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FROM  THE 
CHAIRMAN  AND 
EXECUTIVE 
EDITOR 

Dr.  Larrie  D.  Ferreiro 


The  theme  for  this  edition  of  the  Defense 
Acquisition  Research  Journal  is  “The 
Method  Matters.”  The  lead  article  is 
“Survey  of  Modular  Military  Vehicles: 
Benefits  and  Burdens,”  by  Jean  M.  Dasch 
and  David  J.  Gorsich,  which  goes  a  long 
way  to  defining  the  often-misunder¬ 
stood  word  “modularity”  and  provides 
a  balanced  look  at  the  benefits  and 
drawbacks  of  this  acquisition  method¬ 
ology.  Acquisition  methodology  is  also 
at  the  heart  of  a  classic  article  from  the  Summer  1995  issue  of  the 
Acquisition  Review  Quarterly,  entitled  “Technology  Approach:  DoD 
Versus  Boeing  (A  Comparative  Study)  ”  by  A.  Lee  Battershell.  The 
author  examines  how  the  market-driven  approach  to  development— 
where  cost  and  schedule  dominate  decision  making— contrasts  with 
the  military’s  performance-driven  approach  and  how  each  one  can 
affect  development  time. 

“Tapping  Transaction  Costs  to  Forecast  Acquisition  Cost  Breaches,” 
by  Laura  E.  Armey  and  Diana  I.  Angelis,  highlights  an  intriguing 
method  to  predict  program  cost  overruns  and  breaches  by  focusing 
on  transaction  costs,  as  measured  by  proxy  indicators  of  systems 
engineering  and  program  management  costs,  as  a  leading  indicator. 
Finally,  “Human  Systems  Engineering  and  Program  Success— 
A  Retrospective  Content  Analysis,”  by  Liana  Algarfn,  describes  the 
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impact  of  early  identification  and  assessment  of  human  systems 
integration  requirements  on  the  development  life-cycle  costs  and 
schedules  of  acquisition  programs. 

Please  note  the  updates  to  the  DAU  Research  Agenda  2016-2017. 
Based  upon  Better  Buying  Power  3.0,  in  the  area  of  “Competition” 
we  have  added  a  substantial  section  titled  “Improve  DoD  Outreach 
for  Technology  and  Products  from  Global  Markets.”  As  always,  the 
full  agenda  is  intended  to  make  researchers  aware  of  the  topics 
that  are,  or  should  be,  of  particular  concern  to  the  broader  defense 
acquisition  community  within  the  federal  government,  academia, 
and  industrial  sectors.  To  view  the  agenda  in  its  entirety,  visit  http:// 
www.dau.mil/research/Pages/researchareas.aspx. 

The  astute  reader  will  note  changes  to  our  masthead.  Richard  T. 
Ginman,  Andre  J.  Gudger,  Dr.  Ned  Kock,  and  James  E.  Thomsen 
have  all  moved  on  to  other  endeavors,  and  we  wish  them  well.  On 
the  other  hand,  we  are  pleased  to  welcome  David  Gallop  to  our 
editorial  board. 

The  featured  book  in  this  issue’s  Defense  Acquisition  Professional 
Reading  List  is  To  Engineer  is  Human:  The  Role  of  Failure  in 
Successful  Design,  by  Henry  Petroski.  This  book  is  frequently  cited 
by  Defense  AT&L  leadership  as  one  ofthe  more  influential  works.  We 
are  grateful  to  Dr.  Petroski  for  graciously  contributing  a  summary 
of  the  book  in  his  own  words. 
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DAU  CENTER 
FOR  DEFENSE 
ACQUISITION 

RESEARCH  AGENDA  2016-2017 


This  Research  Agenda  is  intended  to  make  researchers  aware  of 
the  topics  that  are,  or  should  be,  of  particular  concern  to  the  broader 
defense  acquisition  community  within  the  federal  government, 
academia,  and  defense  industrial  sectors.  The  center  compiles  the 
agenda  annually,  using  inputs  from  subject  matter  experts  across 
those  sectors.  Topics  are  periodically  vetted  and  updated  by  the 
DAU  Center’s  Research  Advisory  Board  to  ensure  they  address 
current  areas  of  strategic  interest. 

The  purpose  of  conducting  research  in  these  areas  is  to  provide 
solid,  empirically  based  findings  to  create  a  broad  body  of  knowl¬ 
edge  that  can  inform  the  development  of  policies,  procedures,  and 
processes  in  defense  acquisition,  and  to  help  shape  the  thought  lead¬ 
ership  for  the  acquisition  community.  Most  of  these  research  topics 
were  selected  to  support  the  DoD’s  Better  Buying  Power  Initiative 
(see  http://bbp.dau.mil).  Some  questions  may  cross  topics  and  thus 
appear  in  multiple  research  areas. 

Potential  researchers  are  encouraged  to  contact  the  DAU  Director 
of  Research  (research@dau.mil)  to  suggest  additional  research 
questions  and  topics.  They  are  also  encouraged  to  contact  the 
listed  Points  of  Contact  (POC),  who  maybe  able  to  provide  general 
guidance  as  to  current  areas  of  interest,  potential  sources  of  infor¬ 
mation,  etc. 
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Competition  POCs 

•  John  Cannaday,  DAU:  john.cannaday@dau.mil 

•  Salvatore  Cianci,  DAU:  salvatore.cianci@dau.mil 

•  Frank  Kenlon  (global  market  outreach),  DAU:  frank. 
kenlon@dau.mil 

Measuring  the  Effects  of  Competition 

•  What  means  are  there  (or  can  be  developed)  to  measure 
the  effect  on  defense  acquisition  costs  of  maintaining 
the  defense  industrial  base  in  various  sectors? 

•  What  means  are  there  (or  can  be  developed)  of  mea¬ 
suring  the  effect  of  utilizing  defense  industrial 
infrastructure  for  commercial  manufacture,  and  in 
particular,  in  growth  industries?  In  other  words,  can 
we  measure  the  effect  of  using  defense  manufacturing 
to  expand  the  buyer  base? 

•  What  means  are  there  (or  can  be  developed)  to  deter¬ 
mine  the  degree  of  openness  that  exists  in  competitive 
awards? 

•  What  are  the  different  effects  of  the  two  best  value 
source  selection  processes  (tradeoff  vs.  lowest  price 
technically  acceptable)  on  program  cost,  schedule,  and 
performance? 

Strategic  Competition 

•  Is  there  evidence  that  competition  between  system 
portfolios  is  an  effective  means  of  controlling  price 
and  costs? 

•  Does  lack  of  competition  automatically  mean  higher 
prices?  For  example,  is  there  evidence  that  sole  source 
can  result  in  lower  overall  administrative  costs  at  both 
the  government  and  industry  levels,  to  the  effect  of 
lowering  total  costs? 

•  What  are  the  long-term  historical  trends  for  compe¬ 
tition  guidance  and  practice  in  defense  acquisition 
policies  and  practices? 
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•  To  what  extent  are  contracts  being  awarded  non- 
competitively  by  congressional  mandate  for  policy 
interest  reasons?  What  is  the  effect  on  contract  price 
and  performance? 

•  What  means  are  there  (or  can  be  developed)  to  deter¬ 
mine  the  degree  to  which  competitive  program  costs 
are  negatively  affected  by  laws  and  regulations  such  as 
the  Berry  Amendment,  Buy  America  Act,  etc.? 

•  The  DoD  should  have  enormous  buying  power  and  the 
ability  to  influence  supplier  prices.  Is  this  the  case? 
Examine  the  potential  change  in  cost  performance 
due  to  greater  centralization  of  buying  organizations 
or  strategies. 

Effects  of  Industrial  Base 

•  What  are  the  effects  on  program  cost,  schedule,  and 
performance  of  having  more  or  fewer  competitors? 
What  measures  are  there  to  determine  these  effects? 

•  What  means  are  there  (or  can  be  developed)  to  measure 
the  breadth  and  depth  of  the  industrial  base  in  various 
sectors  that  go  beyond  simple  head-count  of  providers? 

•  Has  change  in  the  defense  industrial  base  resulted  in 
actual  change  in  output?  How  is  that  measured? 

Competitive  Contracting 

•  Commercial  industry  often  cultivates  long-term,  exclu¬ 
sive  (noncompetitive)  supply  chain  relationships.  Does 
this  model  have  any  application  to  defense  acquisition? 
Under  what  conditions/circumstances? 

•  What  is  the  effect  on  program  cost,  schedule,  and 
performance  of  awards  based  on  varying  levels  of 
competition:  (a)  “Effective”  competition  (two  or  more 
offers);  (b)  “Ineffective”  competition  (only  one  offer 
received  in  response  to  competitive  solicitation);  (c) 
split  awards  vs.  winner  take  all;  and  (d)  sole  source. 
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Improve  DoD  Outreach  for  Technology  and  Products 
from  Global  Markets 

•  How  have  militaries  in  the  past  benefitted  from  global 
technology  development? 

•  How/why  have  militaries  missed  the  largest  techno¬ 
logical  advances? 

•  What  are  the  key  areas  that  require  the  DoD’s  focus  and 
attention  in  the  coming  years  to  maintain  or  enhance 
the  technological  advantage  of  its  weapon  systems  and 
equipment? 

•  What  types  of  efforts  should  the  DoD  consider  pursu¬ 
ing  to  increase  the  breadth  and  depth  of  technology 
push  efforts  in  DoD  acquisition  programs? 

•  How  effectively  are  the  DoD’s  global  science  and  tech¬ 
nology  investments  transitioned  into  DoD  acquisition 
programs? 

•  Are  the  DoD’s  applied  research  and  development  (i.e., 
acquisition  program)  investments  effectively  pursuing 
and  using  sources  of  global  technology  to  affordably 
meet  current  and  future  DoD  acquisition  program 
requirements?  If  not,  what  steps  could  the  DoD  take 
to  improve  its  performance  in  these  two  areas? 

•  What  are  the  strengths  and  weaknesses  of  the  DoD’s 
global  defense  technology  investment  approach  as 
compared  to  the  approaches  used  by  other  nations? 

•  What  are  the  strengths  and  weaknesses  of  the  DoD’s 
global  defense  technology  investment  approach  as 
compared  to  the  approaches  used  by  the  private  sec¬ 
tor— both  domestic  and  foreign  entities  (companies, 
universities,  private -public  partnerships,  think  tanks, 
etc.)? 

•  How  does  the  DoD  currently  assess  the  relative  benefits 
and  risks  associated  with  global  versus  U.S.  sourcing 
of  key  technologies  used  in  DoD  acquisition  programs? 
How  could  the  DoD  improve  its  policies  and  procedures 
in  this  area  to  enhance  the  benefits  of  global  technology 
sourcing  while  minimizing  potential  risks? 
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•  How  could  current  DoD/U.S.  Technology  Security  and 
Foreign  Disclosure  (TSFD)  decision-making  policies 
and  processes  be  improved  to  help  the  DoD  better  bal¬ 
ance  the  benefits  and  risks  associated  with  potential 
global  sourcing  of  key  technologies  used  in  current  and 
future  DoD  acquisition  programs? 

•  How  do  DoD  primes  and  key  subcontractors  currently 
assess  the  relative  benefits  and  risks  associated  with 
global  versus  U.S.  sourcing  of  key  technologies  used  in 
DoD  acquisition  programs?  How  could  they  improve 
their  contractor  policies  and  procedures  in  this  area 
to  enhance  the  benefits  of  global  technology  sourcing 
while  minimizing  potential  risks? 

•  How  could  current  U.S.  Export  Control  System  deci¬ 
sion-making  policies  and  processes  be  improved  to 
help  the  DoD  better  balance  the  benefits  and  risks 
associated  with  potential  global  sourcing  of  key  tech¬ 
nologies  used  in  current  and  future  DoD  acquisition 
programs? 

Comparative  Studies 

•  Compare  the  industrial  policies  of  military  acquisition 
in  different  nations  and  the  policy  impacts  on  acquisi¬ 
tion  outcomes. 

•  Compare  the  cost  and  contract  performance  of  highly 
regulated  public  utilities  with  nonregulated  “natural 
monopolies,”  e.g.,  military  satellites,  warship  building, 
etc. 

•  Compare  contracting/competition  practices  between 
the  DoD  and  complex,  custom-built  commercial  prod¬ 
ucts  (e.g.,  offshore  oil  platforms). 

•  Compare  program  cost  performance  in  various  market 
sectors:  highly  competitive  (multiple  offerors),  limited 
(two  of  three  offerors),  monopoly? 

•  Compare  the  cost  and  contract  performance  of  mil¬ 
itary  acquisition  programs  in  nations  having  single 
“purple”  acquisition  organizations  with  those  having 
Service-level  acquisition  agencies. 


XVI 


t  ~  ^ 

76 


ISSUE 
JANUARY  2016 
VOL.  23  NO.  1 


We’re  on  the  Web  at: 

http://www.dau.mil/pubscats/Pages/ARJ.aspx 

and 

http://dau.dodlive.mil 


1 


Survey  of  Modular 
Military  Vehicles: 

BENEFIT 


and  BURDENS 


Jean  M.  Dasch  and  David  J.  Gorsich 


Modularity  in  military  vehicle  design  is  generally  considered 
a  positive  attribute  that  promotes  adaptability  resilience, 
and  cost  savings.  The  benefits  and  burdens  of  modularity 
are  considered  by  studying  historical  programs  dating 
back  to  World  War  II.  Using  a  taxonomy  developed  at  the 
U.S.  Army  Tank  Automotive  Research,  Development  and 
Engineering  Center,  vehicles  were  considered  based  on 
horizontal  modularity,  vertical  modularity,  and  distributed 
modularity.  Examples  were  given  for  each  type,  including 
the  most  extensive  attempt  at  horizontal  modularity  in  the 
1980s,  known  as  the  Armored  Family  of  Vehicles.  Following 
these  examples,  various  cost/benefit  studies  over  the  life 
cycle  of  the  vehicle  are  reviewed  with  differing  conclusions 
depending  on  the  initial  assumptions.  Finally,  a  number  of 
design  factors  are  included  that  should  be  considered  in 
any  program  on  modular  vehicles,  as  well  as  some  recent 
initiatives  that  guide  the  path  forward. 


Keywords:  military  vehicles,  modularity,  modular,  armored  family  of 
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Modularity  and  adaptability  are  considered  desirable,  cost-saving  attri¬ 
butes  for  military  vehicles.  Army  Chief  of  Staff  Raymond  Odierno  has  stated 
that,  “We  need  to  become  more  agile,  adaptable,  and  responsive  to  a  wide 
variety”  of  threats  (Ackerman,  2011,  para.  9).  Modularity  also  corresponds 
with  an  Army  definition  of  a  resilient  system  as  one  that  “is  easily  adapted  to 
many  others  through  reconfiguration  or  replacement”  (Holland,  2013,  p.  3). 
Finally,  the  expected  cost  benefits  fit  well  with  the  DoD  initiative  promoting 
Better  Buying  Power,  which  emphasizes  the  need  for  Modular  Open  Systems 
Architecture  in  program  planning. 

But  the  definition  of  modularity  can  cover  a  large  design  range.  On  one  end 
of  the  spectrum,  the  purpose  of  modularity  is  to  lower  costs  by  using  com¬ 
mon  components  or  a  common  chassis  over  a  number  of  vehicle  variants. 
On  the  other  end  of  the  spectrum,  modular  vehicles  are  those  that  can  be 
interchanged  in  theater  to  accomplish  different  purposes.  Thus,  modularity 
may  have  different  end  states/cost  savings  due  to  commonality  or  greater 
operational  flexibility  to  perform  multiple  missions. 

Past  attempts  at  modularity  since  World  War  II  will  be  reviewed  throughout 
this  article.  Historically,  modularity  usually  involved  producing  variants  on 
a  single  chassis,  defined  as  horizontal  modularity  in  a  subsequent  section 
of  this  article.  Newer  attempts  at  modularity  incorporate  more  ambitious 
designs  that  strive  for  agile  and  adaptable  vehicles.  Following  the  examples, 
past  studies  on  the  benefits  and  burdens  of  modularity  over  a  vehicle’s  life 
cycle  will  be  reviewed. 

The  ultimate  question  of  whether  a  new  system  should  be  modularized 
cannot  be  answered  here  due  to  the  many  variables  at  play.  However,  this 
article  will  lay  out  some  of  the  past  successes  and  failures,  and  provide 
design  points  for  consideration.  Finally,  two  current  efforts  to  determine 
modularity  benefits  and  burdens  for  specific  scenarios  will  be  described- 
one  through  a  U.S.  Army  Tank  Automotive  Research,  Development  and 
Engineering  Center  (TARDEC) -sponsored  Automotive  Research  Center 
(ARC)  project,  and  the  other  through  the  Office  of  Naval  Research  (ONR). 


Definitions  of  Modular  Systems 

As  mentioned  earlier,  modularity  can  include  a  wide  range  of  vehicle 
platforms  that  can  fulfill  a  variety  of  missions.  Modular  vehicles  might  be 
considered  the  opposite  of  unique  vehicles  in  which  each  vehicle  is  designed 
and  built  for  a  specific  purpose.  Somewhere  between  unique  and  modular 
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lies  the  concept  of  commonality  in  which  different  vehicle  platforms  share 
common  components.  Commonality  for  military  equipment  was  explored  by 
a  Rand  Corporation  study  sponsored  by  the  Army  Capabilities  Integration 
Center  (Held,  Newsome,  &  Lewis,  2008).  As  expected,  commonality  was 
found  to  be  desirable,  for  it  can  increase  operational  flexibility  and  reduce 
procurement,  logistical,  and  training  burdens.  But,  perhaps  less  expected, 
it  can  also  decrease  design  freedom  and  may  increase  costs  due  to  excess 
functionality.  So  it  is  with  modularity— the  perceived  benefits  of  modularity 
can  sometimes  be  offset  by  the  negative  aspects. 

Prior  to  discussing  the  benefits  and  burdens  of  modularity,  it  is  necessary 
to  define  modularity  as  used  in  this  article.  The  Advanced  Concepts  Team 
at  TARDEC  investigated  and  presented  a  taxonomy  of  modular  approaches 
(Iler,  2009)  based  on  ideas  developed  in  the  1990s  that  guided  the  thinking 
during  the  pre-Future  Combat  Systems  (FCS)  era.  The  three  types  of  mod¬ 
ularity  in  this  taxonomy  are  horizontal,  vertical  (articulated  or  not),  and 
distributed  (Figure  1).  The  vertical/horizontal  designation  refers  to  the 
orientation  of  the  mating  surface  between  the  modules.  The  three  types 
of  modularity  are  further  described  in  subsequent  sections  of  this  article. 


Source:  Iler,  2009 

Note.  C4I  =  Command,  Control,  Communications,  Computers,  and  Intelligence. 
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Horizontal  Modularity 
(Family  of  Vehicles) 

By  far  the  most  common  form  of  modularity  to  date  is  that  of  horizontal 
modularity,  also  described  as  a  Family  of  Vehicles  (FOV).  An  FOV  usually 
shares  a  common  chassis  including  a  powertrain,  suspension,  and  wheels  or 
tracks  with  a  unique  “box”  on  top  consisting  of  associated  mission-specific 
equipment.  This  type  of  modularity  was  used  during  World  War  II  when 
the  chassis  from  the  M4  Sherman  tank  was  used  for  self-propelled  artillery, 
tank  destroyers,  and  numerous  tracked  carriers  (Hunnicutt,  1990,  p.  281). 
Other  examples  include  the  M60  series  of  tanks  and  the  M107,  M108,  M109, 
and  M110  artillery  systems  in  the  1950s.  In  general,  the  variants  come  from 
a  production  facility  and  are  not  interchangeable  in  theater.  Probably  the 
most  ambitious  examples  of  an  entire  FOV  being  designed  simultaneously 
are  the  Armored  Family  of  Vehicles  (AFV)  from  the  1980s  (Lopez,  1987) 
and  the  FCS  from  the  2000s. 

Armored  Family  of  Vehicles— 1980s 

The  goal  of  the  AFV  was  to  have  a  common  chassis  and  components 
integrated  with  various  mission  modules.  A  minimum  number  of  chassis 
and  a  maximum  number  of  common  system  components  were  planned. 
“Reduction  in  cost  will  be  achieved  through  modularity,  component  com¬ 
monality,  and  multiple  systems  capabilities  combined  so  as  to  achieve 
required  effectiveness  with  more  survivable,  cost-effective  systems”  (Sunell 
et  al,  1987,  p.  VI-7).  Army  studies  had  indicated  that  using  a  common  chas¬ 
sis  and  common  components  could  reduce  future  operational  and  support 
(O&S)  costs. 

Originally,  29  different  mission  modules  were  to  be  built  on  four  weight- 
class  platforms  (Armored  Vehicle  Technologies  Associated  [AVTA],  1988, 
p.  578).  Due  to  the  high  costs  involved  in  developing  so  many  systems,  the 
number  of  variants  kept  dropping  until  the  program  was  eventually  scaled 
back  to  four  vehicles  on  a  heavy  chassis  and  two  on  a  medium  chassis  (ASM 
[Armored  System  Modernization]  Program,  2013).  The  four  variants  on  the 
heavy  chassis  included  the  Block  III  tank,  the  Future  Infantry  Fighting 
Vehicle,  the  Advanced  Field  Artillery  System,  and  the  Combat  Mobility 
Vehicle  (Figure  2).  The  chassis  would  have  common  elements  such  as 
engine,  transmission,  suspension,  modular  armor,  and  tracks.  The  Block  III 
tank  was  given  the  highest  priority.  The  downsized  program  was  renamed 
the  Heavy  Force  Modernization  Program.  Risk  was  to  be  minimized  by 
designing  systems  for  optimum  commonality  and  modularity  (Boelke, 
1992). 
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FIGURE  2.  FOUR  VARIANTS  TO  BE  BUILT  ON  THE  ARMORED  FAMILY 
OF  VEHICLES  HEAVY  CHASSIS 


Block  III  Tank 


Advanced  Field  Artillery  System 


Future  Infantry  Fighting  Vehicle 


Combat  Mobility  Vehicle 


Source:  GAO,  1991 


In  1991,  the  General  Accounting  Office  (GAO)  evaluated  the  program.  It  argued 
that  the  Army  was  using  the  Soviet  threat  as  the  justification  for  starting  with 
the  Block  III  tank,  whereas  the  Soviet  threat  had  diminished  considerably.  Due 
to  this  and  the  high  costs  of  the  program  ($59  billion)  in  times  of  decreasing 
defense  budgets,  the  GAO  suggested  further  evaluation  before  proceeding 
(GAO,  1991).  Eventually,  the  AF  V  was  not  implemented  although  versions  of 
the  field  artillery  system  survived  into  the  2000s  (e.g.,  Crusader). 

Future  Combat  Systems— Recent  Example  of  an  Armored 
Family  of  Vehicles 

The  FCS  program  of  this  century  (Kotchman,  2004)  has  many  com¬ 
monalities  with  the  AFV  of  the  1980s.  The  FCS  Manned  Ground  Vehicle 
(MGV)  program  would  have  replaced  the  Army’s  heavy  tracked  vehicles 
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developed  in  the  1960s  and  1970s  with  lighter,  more  mobile  combat  vehi¬ 
cles.  MGV  included  eight  variants  of  tracked  vehicles  built  on  a  common 
chassis:  a  mounted  combat  system,  a  reconnaissance  and  surveillance 
vehicle,  a  non-line-of-sight  cannon,  a  non-line-of-sight  mortar,  a  recovery 
and  maintenance  vehicle,  an  infantry  carrier  vehicle,  a  command  and  con¬ 
trol  vehicle,  and  a  medical  vehicle.  The  common  chassis  included  the  crew 
station,  propulsion,  vetronics,  and  suspension.  They  touted  commonality  in 
development,  training,  maintenance,  tools,  logistics  approach,  production, 
and  computing.  In  all,  over  70  percent  of  the  components  were  designed  for 
commonality  among  the  variants  (Zanini,  2009). 

As  described  in  a  RAND  report,  FCS  was  the  “largest  and  most  ambi¬ 
tious  planned  acquisition  program  in  the  Army’s  history”  (Pernin  et  al., 
2012,  p.  xvii).  However,  in  April  2009  then-Secretary  of  Defense  Robert 
Gates  cancelled  most  of  the  program.  Many  reasons  have  been  given  for  the 
cancellation,  which  were  captured  in  the  RAND  report  and  a  subsequent 
Government  Accountability  Office  (2009)  report.  As  the  vehicle  weight 
increased  from  the  original  19  to  30  tons  due  to  increasing  requirements,  the 
common  chassis  had  to  be  replaced  with  three  different  chassis.  As  with  the 
1980s’  program,  the  reasons  for  cancellation  were  not  directly  attributable 
to  the  modular  approach. 

Horizontal  Modularity/Common  Chassis— Vehicles 
in  Production 

Despite  the  cancellation  of  the  ambitious  AFV  and  the  FCS  programs, 
many  other  military  vehicles  have  evolved  into  families,  such  as  the  Bradley, 
Abrams,  Stryker,  and  many  others  (Science  Applications  International 
Corporation  [SAIC],  2008).  In  most  cases,  the  families  were  not  planned 
from  the  outset,  but  rather  a  unique  vehicle  was  designed  and  built  for  a 
single  purpose.  Later  the  original  chassis  was  used  for  variants.  This  had  the 
advantage  of  lower  initial  Research,  Development,  Testing  and  Evaluation 
(RDT&E)  costs  in  that  only  requirements  for  a  single  vehicle  needed  to  be 
considered  rather  than  trying  to  design  a  chassis  to  fit  many  roles.  On  the 
downside,  the  original  chassis  might  not  meet  the  requirements  for  vehicles 
planned  later.  The  following  discussion  cites  four  examples  of  horizontal 
modularity. 

The  first  example  is  the  Stryker  FOV  (Figure  3),  which  was  designed  to  allow 
fast  deployment  and  to  fill  the  capability  gap  between  the  heavily  armored 
Abrams  and  Bradley,  and  the  lightly  armored  High  Mobility  Multi-Purpose 
Wheeled  Vehicle  (HMMWV).  The  Stryker,  an  8-wheeled  fighting  vehicle, 
has  been  produced  by  General  Dynamics  Land  Systems  since  2005.  It  comes 
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in  several  variants  that  share  a  common  engine,  transmission,  hydraulics, 
wheels,  tires,  differential,  and  transfer  case.  Variants  include  an  infantry 
carrier  vehicle;  a  reconnaissance  vehicle;  a  mobile  gun  system;  a  mortar 
carrier;  a  command  vehicle;  a  fire  support  vehicle;  an  engineer  squad  vehi¬ 
cle;  a  medical  evacuation  vehicle;  an  anti-tank  guided  missile  vehicle;  and  a 
nuclear,  biological,  and  chemical  reconnaissance  vehicle  (army-technology, 
com,  n.d.;  Stryker,  2015). 


Another  example,  created  by  the  Finnish  Defense  Industry,  is  the  8x8  Patria 
Armored  Modular  Vehicle  (AMV)  (army-technology.com,  n.d.,  Patria; 
Patria  AMV,  2015),  which  allows  the  incorporation  of  different  turrets, 
sensors,  and  communication  systems  on  the  same  carriage  (17  to  30  tons) 
(Figure  4).  There  are  three  platforms:  the  basic  platform,  a  high-roof  plat¬ 
form,  and  a  heavy-weapon  platform.  The  basic  platform  can  accommodate 
an  armored  personnel  carrier;  infantry  fighting  vehicle;  command  and 
control;  and  ambulance,  reconnaissance,  and  other  vehicle  variants.  The 
high-roof  platform  is  suitable  for  command,  control,  and  communications; 
large  ambulance;  and  workshop  vehicles.  The  heavy -weapon  platform  has 
a  stronger  structure  for  heavy-weapon  systems.  The  AMV  is  used  in  seven 
countries  and  can  carry  a  14-ton  payload. 
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In  a  few  cases,  horizontal  modularity  changeover  can  be  accomplished 
in  the  field,  such  as  with  the  Boxer,  a  multirole  armored  fighting  vehicle 
(Boxer,  2015).  The  result  of  a  German-Dutch  collaboration,  the  Boxer  is  a 
very  large,  33-ton  vehicle  that  has  a  number  of  mission  modules  that  can  be 
interchanged  in  the  field  to  accomplish  different  goals  (Figure  5).  Mission 
modules  can  be  interchanged  in  an  hour  to  create  an  armored  personnel 
carrier,  an  infantry  fighting  vehicle,  or  an  ambulance  among  others. 
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Another  recent  example  of  a  modular  vehicle  that  can  be  interchanged  in  the 
field  is  the  Cameleon  IV440  Modular  Mission  System  (Figure  6)  developed  by 
OVIK  in  the  United  Kingdom  (OVIK  Cameleon,  2015;  OVIK  Crossway,  n.d.). 
It  was  originally  designed  in  2010  for  a  4x4  chassis  with  modules  for  defense 
or  commercial  applications.  Since  then,  it  has  expanded  to  heavier  systems 
and  is  now  available  on  platforms  ranging  from  5.5  to  40  tons.  Seven  modules 
are  available,  including  patrol  vehicles,  weapons  platforms,  fuel  bowsers, 
and  power  generators,  with  many  more  in  development.  The  base  platform 
includes  a  hydraulic  system  to  enable  rapid  platform  reconfiguration.  Module 
changeover  can  be  accomplished  by  one  man  in  only  1  minute.  The  modules 
are  expected  to  last  three  times  longer  than  the  base  vehicles. 


FIGURE  6.  CAMELEON  DE-MOUNTABLE  FUEL  BOWSER  WITH 
AMBULANCE  AND  PATROL  MODULE  IN  REAR 


Source:  OVIK  Cameleon,  2014 


Vertical  Modularity 

In  this  modular  construct  (see  Figure  1),  two  units  are  joined  together, 
either  directly  or  articulated.  The  two  units  would  typically  have  different 
purposes,  and  each  could  carry  some  combination  of  crew,  power  unit,  and 
mission  equipment.  The  separate  units  could  be  independent  or  dependent. 

As  part  of  the  AF  V  program,  a  side  study  was  conducted  on  coupled  vehicles 
(Figure  7),  in  which  a  pair  of  armored  vehicles  was  considered  as  a  substitute 
for  the  Ml  tank  (Schwartz,  1988).  A  large  part  of  the  motivation  was  to  lower 
the  weight  of  individual  units  compared  to  the  Ml  tank;  each  unit  would  be 
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less  than  40  tons.  Further  modularity  could  be  added  by  having  two  different 
weight  units;  for  instance,  a  20-ton  unit  and  a  30-ton  unit  could  be  used  singly 
or  in  pairs  to  produce  20-,  30-,  40-,  50-,  or  60 -ton  vehicles. 


FIGURE  7.  COUPLED  TANKS  WITH  HARD  AND  SOFT  COUPLING 


In  the  original  nomenclature,  the  two  units  could  be  hard-coupled  or 
soft-coupled  with  a  communication  link  allowing  greater  separation 
between  units.  Both  units  would  be  independently  powered.  The  front  vehi¬ 
cle  would  carry  the  main  gun  and  ammunition.  The  second  vehicle  would 
carry  the  crew,  a  secondary  armament,  fire  control,  and  control  and  com¬ 
munications  equipment.  Schwartz  (1988)  saw  survivability  advantages  in 
several  areas:  the  rear  crew  vehicle  would  be  protected  by  the  front  vehicle 
from  enemy  fire  or  mines;  the  crew  would  be  separated  from  the  munitions; 
if  the  front  vehicle  were  disabled,  the  rear  vehicle  could  separate  and  return 
to  base;  and  finally,  the  crew  vehicle  could  operate  at  a  safe  standoff  distance 
in  the  case  of  soft  coupling.  Although  the  total  footprint  of  a  coupled  tank 
would  be  greater,  each  part  would  be  smaller,  narrower,  and  less  weighty 
than  an  Ml,  allowing  for  easier  transport.  The  lower  weight  would  also 
translate  into  the  requirement  for  less  powerful  powertrains  and  lighter 
tracks,  suspension,  etc.  In  many  cases,  improved  mobility  would  result  in 
several  advantages:  the  lower  weight  would  lead  to  lower  ground  pressures; 
the  two  units  could  push  or  pull  each  other  if  hard  coupled;  bridges  and 
roadways  with  lower  load  limits  would  be  accessible;  and  recovery  vehicles 
could  be  lighter. 
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In  the  1960s,  TARDEC  designed  a  concept  vehicle  with  vertical  modularity, 
the  Twister  (Figure  8),  which  was  built  by  Lockheed  Missiles  and  Space 
Corporation  (Dasch  &  Gorsich,  2012).  Each  section  had  four  wheels  and 
its  own  engine.  A  crew  of  three  rode  in  the  back.  It  had  a  unique  pivot  yoke 
between  the  two  sections,  which  provided  pitch  articulation  to  the  front 
and  yaw  and  roll  between  the  two  sections,  allowing  it  to  climb  over  3-foot 
walls.  The  rear  body  wheels  were  mounted  in  tandem  pairs  on  individual 
center-pivot,  powered,  sprung  walking  beams,  which  permitted  a  total 
wheel  travel  of  27  inches.  Other  innovations  included  low-pressure  radial 
ply  tires  and  power  disc  brakes— both  cutting-edge  technologies  for  the  time. 
It  could  cover  rough  ground  at  an  amazing  65  miles  per  hour.  However,  it 
never  went  beyond  the  prototype  stage  due  to  its  complexity,  cramped  crew 
quarters,  and  the  traditional  track  mentality  (Wynbelt,  1972). 


In  the  1970s,  TARDEC  experimented  further  with  vertical  modularity  using 
articulated  vehicles;  the  goal  was  less  toward  modularity  and  more  toward 
improvements  in  off-road  mobility.  Two  M113s  were  connected  through  a 
cybernetically  controlled  articulation  joint  invented  by  TARDEC,  which 
featured  positive  pitch  and  yaw  control  with  roll  freedom  that  provided  force 
feedback  to  the  operator  (Figure  9).  Two  hydraulic  cylinders  mounted  on  the 


Defense  ARJ,  January  2016,  Vol.  23 No.  1 : 2-27 


13 


A  Publication  of  the  Defense  Acquisition  University 


http://www.dau.mil 


vehicles  provided  the  force  to  control  the  relative  pitch  and  yaw  between  the 
two  coupled  M113s.  A  single  M113  could  climb  an  18-inch  high  obstacle,  but 
the  two  articulated  M113s  could  climb  a  5-foot  wall.  The  articulated  vehicle 
could  also  cross  open  trenches  up  to  10  feet  wide;  climb  a  60  percent  slope; 
enter,  cross,  and  exit  a  waterway;  and  be  controlled  from  either  the  front  or 
back  vehicle  by  means  of  a  joystick  (Beck  &  Kamm,  1974;  1975). 


A  TARDEC  team  is  currently  involved  in  a  concept  development  and 
analysis  project  for  a  modular  tactical  truck,  known  as  the  Joint  Tactical 
Transportation  System  (JTTS).  The  team  is  evaluating  a  medium/heavy 
tactical  truck  system  that  can  be  converted  from  a  6x6  to  an  8x8  to  a  10x10 
truck  in  the  field  through  the  addition  of  extra  wheels,  axles,  and  associated 
hardware  (Figure  10).  This  unique  modular  concept  is  facilitated  through 
the  use  of  a  hybrid  propulsion  system  and  in-line  motors  on  each  wheel. 
When  equipped  with  a  load  handling  system  to  interchange  payloads,  this 
concept  has  elements  of  both  vertical  and  horizontal  modularity.  If  JTTS 
is  realized,  eight  vehicle  families,  including  12  different  chassis,  can  be 
reduced  to  a  single  system. 
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Distributed  Modularity 

In  this  final  form  of  modularity,  functions  are  distributed  among  dif¬ 
ferent  units  connected  through  a  communications  network  (see  Figure  1). 
Since  some  of  the  units  are  typically  unmanned,  there  are  survivability 
benefits  for  the  manned  control  vehicles,  which  can  operate  at  a  standoff. 
The  unmanned  units,  which  require  less  armor,  enjoy  enhanced  mobility 
due  to  the  reduced  weight. 

An  example  of  distributed  modularity  from  the  past  was  the  Robotic 
Command  Center  (RCC)  developed  by  TARDEC  (Taylor,  1992,  pp.  41-42). 
The  goal  was  to  allow  a  few  personnel  to  control  a  large  number  of  unmanned 
vehicles  while  the  control  center  was  on  the  move.  The  RCC  was  completed 
in  1992  and  consisted  of  a  control  module  mounted  on  a  carrier.  The  module 
carried  a  commander  and  two  driver-operators.  Each  operator  could  control 
two  robotic  vehicles  (HMMWVs).  This  was  the  first  successful  demonstra¬ 
tion  of  multiple  vehicle  control. 

The  Main  Battle  System  (MBS)  is  a  recent  TARDEC  concept  that  utilizes 
distributed  modularity  (Effinger  &  Parker,  2013;  Parker  &  Scott,  2013).  The 
goal  is  to  replace  a  70-ton  M1A2  Abrams  tank  with  three  vehicles:  a  25-  to 
30-ton  manned  vehicle  with  a  crew  of  four  and  two  15-  to  20-ton  unmanned 
vehicles,  each  with  a  120mm  cannon.  The  crew  of  four  would  include  a 
vehicle  operator,  a  commander,  and  two  unmanned  vehicle  operators.  The 
manned  crew  vehicle  would  not  need  to  absorb  the  firing  load  of  the  can¬ 
non  or  the  weight  of  the  ammunition.  The  unmanned  units  would  not  need 
crew-protective  armor. 


Defense  ARJ,  January  2016,  Vol.  23 No.  1 : 2-27 


15 


A  Publication  of  the  Defense  Acquisition  University 


http://www.dau.mil 


As  shown  in  Figure  11,  an  MBS  Platoon  consisting  of  two  supervision  vehi¬ 
cles  and  four  unmanned  direct-fire  vehicles  could  be  used  to  replace  four 
Ml  Abrams  tanks.  Total  crew  would  decrease  from  16  to  8  persons.  Four  Ml 
Abrams  would  have  a  weight  of  -280  tons,  whereas  the  two  MBS  units  would 
have  a  weight  range  of  110  to  140  tons— a  considerable  weight  savings.  In 
addition,  distributing  the  weight  over  six  vehicles  rather  than  four  has  addi¬ 
tional  transport  and  mobility  advantages.  The  supervision  vehicles  could  be 
at  considerable  standoff  from  the  fighting  vehicles  for  increased  survivability. 


Figure  41.  Main  Battle  System  Platoon 


Source:  Department  of  Defense 


Previous  sections  in  this  article  described  the  three  types  of  platform 
modularity— horizontal,  vertical,  and  distributed— with  examples  of  each. 
The  most  common  is  horizontal  modularity,  also  known  as  an  FOV.  The 
next  section  will  review  various  studies  where  the  authors  quantitatively 
estimated  the  benefits  and  burdens  of  modularity. 


Benefits  and  Burdens 

Various  attempts  have  been  made  to  quantify  the  costs  and  benefits 
of  the  use  of  modular  systems.  In  this  section,  the  authors  review  seven 
studies,  of  which  four  are  from  the  era  of  the  AFV.  Due  to  the  envisioned 


16 


Defense ARJ,  January  2016,  Vol.  23 No.  1 : 2-27 


January  2016 


size  of  the  AFV  program,  considerable  effort  went  into  evaluating  costs. 
The  final  three  studies  are  from  analyses  of  more  modern  modular  vehicles 
and  concepts. 

The  costs/benefits  of  the  AFV  were  evaluated  by  the  Advanced  Vehicle 
Testing  Activity  (AVTA)  made  up  of  Food  Machinery  Corporation,  General 
Dynamics,  and  smaller  companies.  They  conducted  a  Life  Cycle  Cost 
Analysis  of  the  AFV  (AVTA,  1988).  The  AVTA  concluded  that  “the  total 
costs  shown  for  the  Development,  Production,  Fielding,  and  Sustainment 
of  the  AFV  Family  of  Vehicles  (Heavy,  Medium,  and  Light,  Wheeled,  and 
Trailer  groups)  provide  a  very  adequate  baseline  for  the  quantification  of 
Life  Cycle  Cost  savings”  (AVTA,  1988,  p.  578).  Most  of  the  savings  came 
from  common  functional  packages  leading  to  savings  in  development,  pro¬ 
curement,  training,  and  support  costs.  Average  savings  from  the  packages 
were  in  the  15-30  percent  range,  distributed  across  the  entire  life  cycle. 
These  are  very  significant  savings  since  the  O&S  costs  typically  dwarf  the 
research  and  design  costs. 

A  second  1988  study  on  battle  damage  assessment  and  repair  found  that  dam¬ 
aged  vehicles  could  be  repaired  and  returned  to  battle  more  quickly  in  the 
case  of  commonality  (Kane,  1988).  A  final  1988  study  found  a  negative  impact 
from  a  modular  AFV  fleet;  it  compared  the  costs  of  combat  service  support, 
supply,  maintenance,  and  transportation  between  an  AFV  fleet  in  theater 
and  a  conventional  armored  fleet  upgraded  to  an  expected  2005  configura¬ 
tion  (Cunningham,  Tollefson,  &  Malcolm,  1988).  They  found  that  the  AFV 
fleet  would  be  more  expensive  in  theater,  primarily  because  the  overall  fleet 
weight  would  be  greater.  When  using  a  single  chassis  for  all  heavy  vehicles, 
the  chassis  has  to  be  big  enough  and  powerful  enough  to  support  the  heaviest 
vehicle  in  the  class.  They  found  the  weight  of  all  heavy  tracked  vehicles  was 
50  percent  greater  in  the  AFV  case,  which  led  to  higher  fuel  usage  and  more 
maintenance  personnel,  as  required,  for  heavier  tracked  vehicles. 

In  the  study  of  coupled  vehicles  discussed  earlier,  a  cost  model  known  as 
TREAD  was  used  to  estimate  production  costs  of  a  fleet  of  coupled  tanks  rel¬ 
ative  to  a  fleet  of  Ml  tanks  (Schwartz,  1988).  Certain  costs  would  be  higher 
such  as  armor  due  to  more  surface  area,  and  sensors  and  communication 
equipment  between  the  two  units.  Other  costs  would  be  lower  due  to  the 
common  chassis  and  the  fact  that  fewer  reserve  units  would  be  required. 
The  final  estimate  was  that  in  a  coupled  fleet  with  less  armor  on  the  front, 
an  unmanned  unit  would  cost  2-5  percent  more  than  a  tank  fleet  unit.  If 
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both  units  were  produced  with  armor  comparable  to  the  Ml,  costs  would  be 
14-19  percent  higher  for  the  coupled  vehicles.  Other  costs  such  as  research 
and  development  (R&D)  and  O&S  were  not  considered. 

During  FCS  Phase  I,  SAIC  evaluated  the  conventional  unique  vehicle  con¬ 
cept  versus  a  modular  concept  for  an  18-ton  platform  (SAIC,  2000).  Using 
TARDEC’s  taxonomy,  they  determined  that  horizontal  modularity  was 
more  efficient  than  vertical  modularity  due  to  poor  mobility  and  surviv¬ 
ability  characteristics  of  independent  modules  and  the  need  for  a  complex 
and  heavy  mating  joint.  However,  the  horizontally  modular  platform  was 
also  not  ideal  because  it  required  a  heavier  hull  structure  to  implement; 
the  structure  would  have  to  be  heavy  enough  to  accommodate  the  largest, 
heaviest  variant,  meaning  that  the  powertrain,  suspension,  tracks,  etc., 
would  be  overdesigned  for  lighter  variants.  In  addition,  the  modules  would 
require  a  ballistic  joint  that  would  be  difficult  to  seal.  Overall,  they  calcu¬ 
lated  that  unique  hulls  are  10-30  percent  more  weight/volume  efficient  than 
a  horizontally  modular  platform. 
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Readers  are  referred  to  an  SAIC  (2008)  report  that  did  a  comprehensive 
investigation  of  acquisition  costs  for  an  FOV  concept,  including  estimated 
acquisition  costs.  They  compiled  a  comprehensive  table  of  the  existing 
“Families  of  Vehicles”  that  includes  the  Bradley,  Abrams,  Light  Armored 
Vehicle,  Stryker,  M113,  HMMWV,  Family  of  Medium  Tactical  Vehicles, 
and  Heavy  Expanded  Mobility  Tactical  Truck.  They  initially  assumed  that 
the  development  phase  for  an  F OV  would  be  burdensome  due  to  the  need  to 
meet  requirements  for  several  variants.  However,  they  learned  that  in  most 
cases,  a  single  unique  vehicle  is  designed  and  built.  Later,  other  variants  that 
make  use  of  the  original  chassis  are  added.  Cost  savings  follow: 

•  An  RDT&E  effort  needed  to  develop  new  variants  is  minimized. 

•  Shared  learning-rate  effect,  economies  of  scale,  and  the 
cost-benefit  of  reduced  training  times  lower  production  costs. 

•  O&S  costs  are  reduced  through  quantity  discounts  and  greater 
economies  of  scale  in  purchasing  parts. 

•  A  high  level  of  commonality  decreases  the  training  needed  for 
maintenance. 

SAIC  estimated  that  a  base  vehicle  for  an  FOV  would  cost  50-100  percent 
more  than  a  unique  vehicle  in  RDT&E  costs.  However,  each  additional  vari¬ 
ant  would  only  require  7.6  percent  of  the  unique  vehicle  costs.  Based  on  the 
SAIC  analysis,  as  more  variants  are  added  the  more  cost-effective  the  FOV 
concept  becomes. 

On  the  other  hand,  SAIC  mentioned  that  the  later  vehicle  plans  sometimes 
had  to  be  abandoned  because  the  original  chassis  could  not  meet  the  expanded 
requirements.  Its  example  was  from  1990,  when  the  Ml  Abrams  chassis  was 
to  be  developed  into  a  tow  vehicle  to  replace  the  M88  recovery  vehicle  and 
an  early  version  of  the  Breacher  Vehicle.  Neither  was  successful  because 
the  Abrams  power  pack  wasn’t  well  suited  to  these  applications.  Another 
example  of  an  unsuccessful  attempt  at  modularity  was  the  self-  propelled 
howitzer,  SP70,  developed  by  the  United  Kingdom,  West  Germany,  and  Italy 
in  the  1970s.  It  made  use  of  a  modified  Leopard  tank  chassis  (Craven,  1983). 
The  collaborative  effort  was  unsuccessful  because  the  chassis  design  forced 
a  complicated  ammunition  handling  system  that  frequently  failed. 

TARDEC’s  Advanced  Concepts  Team  considered  the  weight  and  space 
penalty  from  a  vertical  modularity  concept  vehicle,  known  as  the  RAVE 
(Iler,  2009).  For  two  10-ton  units,  they  estimated  that  the  coupling  unit,  the 
batteries,  and  the  robotic  components  would  add  39  ft3  and  3,940  lbs.  over  a 
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unique  vehicle— an  increase  of  about  10  percent  in  weight.  In  addition,  two 
structural  walls  were  needed  at  the  connection  point,  whereas  one  or  none 
are  needed  in  a  unique  vehicle. 

Table  1  summarizes  the  studies  described  in  this  section.  Some  studies 
were  evaluating  only  one  facet  of  modularity  such  as  repair  costs.  Overall, 
three  studies  found  a  net  benefit,  three  found  a  net  burden,  and  one  found  it 
could  go  either  way  depending  on  the  assumptions.  So  many  variables  are 
affected  by  modularity  during  the  different  phases  of  design,  engineering, 
production,  usage,  repair,  and  sustainment  that  a  definitive  answer  will 
depend  on  circumstances. 


TABLE  1.  VARIOUS  MODULARITY  STUDIES  AND 

DECISION  AS  TO  BENEFIT  OR  BURDEN 

Vehicle  Group 

Reference 

Benefit  or 
Burden 

Comments 

AFV 

AVTA 

Benefit 

15-30%  savings 
in  development, 
procurement,  support 

AFV  repair 

Kane 

Benefit 

Due  to  commonality 

AFV  Cost  in 
Theater 

Cunningham, 
Tollefson,  & 
Malcolm 

Burden 

Higher  fleet  weight  due  to 
common  chassis 

Coupled  Vehicles 

Schwartz 

Benefit  or 
Burden 

Depends  on  whether  front 
vehicle  is  fully  armored 

FCS  Phase  1 

SAIC 

Burden 

Overdesign;  parasitic  hull 
weight 

Families  of 
Vehicles 

SAIC 

Benefit 

Reduced  costs  in  design, 
commonality,  economies 
of  scale 

RAVE  weight 
analysis 

TARDEC 

Burden 

10%  increase  in  weight 

Summary  and  Discussion 

Based  on  the  taxonomy  developed  by  TARDEC’s  Advanced  Concepts 
Team,  three  types  of  modularity  were  considered:  horizontal,  vertical,  and 
distributed.  By  far,  the  most  common  today  is  horizontal,  also  described  as 
an  FOV,  in  which  a  common  chassis  and  powertrain  are  used  for  a  number 
of  variants.  An  important  aspect  of  structural  modularity  is  the  ease  of 
changing  from  one  vehicle  variant  to  another.  Can  it  be  done  in  the  field, 
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or  at  a  depot,  or  only  in  a  production  facility?  In  the  vast  majority  of  cases, 
these  variants  are  purchased  as  is  from  the  manufacturer  and  are  not  inter¬ 
changeable  in  the  field.  A  summary  of  the  advantages  and  disadvantages  of 
each  type  of  modularity  is  captured  in  Table  2. 


TABLE  2.  ADVANTAGES  AND  DISADVANTAGES  OF 

VARIOUS  FORMS  OF  MODULARITY 

Type  of  Modularity 

Advantages 

Disadvantages 

Horizontal 

•  Reduced  design  for 

•  Designing  entire  family 

Modularity 

later  vehicles 

from  scratch  may  be 

•  Commonality  is  built  in 

impossible 

•  Variants  usually  from 
production  facility 

•  Must  be  overdesigned 
to  accommodate 
largest  variant 

Vertical  Modularity 

•  Decreased  weight  of 

•  Overall  weight  greater 

each  module 

•  Some  parts  must  be 

•  More  transportable 

duplicated  in  each 

•  Reconfigurable  in  field 

•  Unmanned  modules 
need  less  protection 

•  Articulated  has  greater 
mobility 

module 

Distributed 

•  Decreased  weight  of 

•  Overall  weight  greater 

Modularity 

each  module 

•  Greatest  need  for  semi- 

•  More  transportable 

autonomous  modules 

•  Reconfigurable  in  field 

•  Some  parts  must  be 

•  Most  flexible  of  all 

duplicated  in  each 

systems 

module 

•  Unmanned  modules 

•  Need  good 

need  less  protection 

communications 

networks 

•  Greatest  complexity 

From  the  previous  discussion,  there  are  obviously  some  advantages  and 
disadvantages  to  modular  vehicles.  Held  et  al.  (2008,  p.  3)  determined  that 
“nuanced  decision  making  is  required”  to  gain  a  significant  benefit  from 
commonality.  The  same  can  be  said  of  modularity.  Some  points  to  consider 
are  captured  in  the  following  discussion: 

Design  Considerations 

•  For  any  modular  vehicle,  design  and  RDT&E  considerations 
will  be  more  extensive  than  for  a  unique  vehicle. 
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•  If  the  chassis  for  a  unique  vehicle  is  used  later  for  additional 
vehicles,  design  and  R&D  requirements  will  be  less  for  suc¬ 
ceeding  variants.  Conversely,  the  original  chassis  might  not 
support  requirements  of  succeeding  variants. 

•  Attempts  to  design  an  entire  FOV  can  easily  become  over¬ 
whelmed  by  competing  requirements  for  the  variants. 

Weight 

•  In  many  forms  of  modularity,  the  total  weight  of  the  modular 
vehicle  will  be  greater  than  that  of  a  unique  vehicle,  due  to 
overdesign  for  some  variants,  parasitic  structure,  and  possible 
extra  sensors,  powertrains,  communication  devices,  etc.,  if  the 
modules  are  capable  of  independent  mobility. 

•  In  the  case  of  vertical  or  distributed  modularity,  the  individ¬ 
ual  modules  each  weigh  less  than  the  total  vehicle,  which  has 
benefits  for  transport  and  mobility. 
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•  In  the  FOV  example,  the  chassis  will  need  to  be  heavy  and 
powerful  enough  to  support  all  variants.  This  overdesign 
requirement  would  increase  vehicle  size/weight  for  some 
variants. 

Adaptability 

•  Since  the  vast  majority  of  today’s  horizontally  modular/ 
common  chassis  vehicles  are  factory-produced,  no  gain  in 
operational  adaptability  occurs. 

•  Vehicle  variants  that  can  be  interchanged  in  the  field  are  excep¬ 
tional  today.  However,  two  recent  examples  were  provided: 
the  German-Dutch  Boxer  and  the  United  Kingdom  Cameleon, 
which  can  be  rapidly  changed  from  one  variant  to  another. 

Costs/Benefits 

•  The  use  of  the  modular  approach  in  vehicles  is  a  potential  solu¬ 
tion  to  address  combat  developer  and  program  requirements 
for  supportability,  cost,  and  adaptable  and  flexible  vehicles. 

•  The  modular  approach  comes  at  a  price  in  trade-offs,  which 
must  be  considered  when  applying  it  to  a  vehicle  development 
effort. 


Path  Forward 

Modularity  undoubtedly  has  strong  potential  benefits  in  lowered  costs 
and  a  more  adaptable  force.  However,  the  survey  of  successful  and  failed 
modularity  attempts  does  not  present  a  clear  picture  to  assess  the  benefits 
of  modularity.  As  quoted  in  a  recent  study,  “the  effect  of  modularity  on  the 
entire  fleet  must  be  considered  with  a  careful  analysis  of  both  performance 
and  life-cycle  cost  implications”  (Bayrak  et  al.,  2015,  p.  2).  Two  studies  have 
recently  been  initiated  to  delve  deeper  into  these  issues. 

TARDEC  has  initiated  a  study  through  the  ARC  to  establish  a  modeling 
framework  for  assessing  the  adaptability  and  costs  of  a  modular  military 
vehicle  fleet  (Bayrak  et  al,  2015).  Their  efforts  center  on  model  simulations 
of  powertrain  demands,  fleet  operations,  transportation  costs,  operating 
costs,  and  acquisition  costs,  with  a  comparison  of  a  modular  fleet  versus 
a  conventional  fleet.  Unlike  earlier  examples,  the  design  of  the  individual 
modular  vehicle  is  not  being  considered.  Rather,  its  modeling  framework 
includes  the  following  items: 
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•  Function  Selection  and  Attributes 

•  Conventional  and  Modular  Fleet  Models 

•  Mobility  and  Fuel:  Powertrain  Models 

•  Fleet  Operation  Models 

•  Cost  Models 

°  Transportation 

°  Acquisition  Considering  Manufacturing 

°  Fleet  Operation 

Building  on  the  ARC  effort,  the  ONR  announced  a  Vehicle  Agnostic 
Modularity  Project  Plan  (VAM,  2014,  p.  28),  with  a  goal  of  understand¬ 
ing  and  demonstrating  modular  vehicles.  The  hypothesis  of  VAM  is  that 
a  well-structured  application  of  modularity  will  “extend  tactical  range, 
extend  operational  reach  and  increase  endurance  in  the  field,  reduce 
excess  capacity  and  reduce  logistic  footprint/burden,  enhance  small  unit 
effectiveness,  lighten  the  Marine  Air- Ground  Task  Force  load,  enhance 
commonality,  and  reduce  Total  Owner  Cost”  (VAM,  2014,  p.  41).  Over  the 
course  of  several  years,  VAM  will  explore  the  benefits  of  vehicle  modularity 
as  well  as  its  challenges. 

An  underlying  premise  of  these  studies  is  that  a  unique  vehicle  geared  to  a 
specific  mission  will  always  outperform  a  modular  vehicle.  However,  over  a 
set  of  varied  missions  over  a  longer  timeframe,  the  modular  fleet  may  offer 
cost  and  agility  benefits  compared  to  a  conventional  fleet.  These  efforts  by 
ONR  and  TARDEC  are  providing  decision  makers  a  framework  to  determine 
the  value  of  modular  fleets  in  operational  environments. 
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Technology  Approach: 
DOD  VERSUS  BOEING 

(A  Comparative  Study) 

A.  Lee  Battershell 


This  is  an  analysis  of  different  approaches  in  the  use  of  technology  by  Boeing 
and  DoD  to  determine  how  they  may  have  affected  development  time  for  the 
C-17  and  the  Boeing  777.  Boeing’s  focus  on  cost,  schedule,  performance,  and 
market  competition  is  contrasted  to  DoD’s  focus  on  performance.  The  paper 
concludes  thatthe  mere  existence  of  atechnology  should  not  obscure  (a)  the 
impact  its  maturity  may  have  on  program  cost  and  risk,  (b)  whether  it  will 
meet  a  real  need  of  the  user  as  opposed  to  a  gold  plated  one,  and  (c)  whether 
the  added  development  time  it  may  require  could  pose  unanticipated  prob¬ 
lems  for  the  customer,  or  even  result  in  fielding  obsolete  weapons  systems. 


Keywords:  C-17  military  cargo  plane,  Boeing  777  passenger  plane,  Air  Force  tactical  plane, 
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The  advantage  we  had  in  Desert  Storm  had  three  major 
components.  We  had  an  advantage  in  people,  an  advantage 
in  readiness,  and  an  advantage  in  technology...  We  need  to 
preserve  that  part  of  the  industrial  base  which  will  give  us  a 
technological  advantage.  (William  Perry,  Secretary  of  Defense) 
(Mercer  &  Roop,  1994) 


What  is  Technology? 


Technology  must  earn  its  way  on  to  a  Boeing  [commercial] 
plane...  In  short,  ourR&D  efforts  will  continue  to  be  custom¬ 


er-driven,  not  technology -driven  (P.  M.  Condit,  President 
of  Boeing,  personal  communication,  November 1994). 

What  are  the  differences  in  the  way  private  indus¬ 
try  and  Government  approach  technology  when 
developing  planes?  Why  does  the  Government  take 
longer  than  the  private  sector  to  develop  a  plane? 


There’s  a  perception  that  high  technology 
included  in  military  planes  contributes  signifi¬ 
cantly  to  the  typical  11  to  21  years  (DiMascio, 

1993)  it  takes  the  Department  of  Defense 
(DoD)  to  develop,  produce,  and  deploy  new  mil¬ 
itary  aircraft.  To  learn  if  it  is  the  technology 
that  takes  so  long,  this  study  explores  the  way 
Boeing  and  the  DoD  approached  technology 
in  developing  the  Boeing  777  and  the  military 
C-17.  One  reason  for  selecting  the  C-17  is  that 
it  does  not  have  the  complex  weapons  systems 
inherent  in  fighters  or  bombers,  and  yet  it  still 
took  more  than  14  years  to  develop  and  deliver.  In 
contrast,  it  took  little  more  than  four  years  to  develop 
and  deliver  an  operational  Boeing  777. 


According  to  Webster’s  Dictionary,  technology  is  defined  as  “The 
applied  science  that  includes  the  study  of  industrial  arts  one  can  apply 
toward  practical  use”  (Guralnik,  1980).  Technology  is  a  method  or  process 
for  handling  a  specific  technical  problem.  By  contrast,  natural  science  is: 
“...the  study  of  knowledge  to  understand  the  nature  of  the  subject  matter 
which  is  being  studied.  Its  purpose  is  for  the  sake  of  understanding— the 
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application  or  usefulness  may  not  be  self  evident  at  that  time.  Technology 
is  the  application  of  scientific  breakthroughs”  (Goldberg,  1995).  When 
one  speaks  of  a  technology  breakthrough,  one  is  defining  a  new  process  or 
method  for  application  of  a  scientific  breakthrough. 


Need  for  Change 


The  DoD  is  coping  with  reduced  resources  and  a  changing  world. 
At  home,  the  American  public  continues  to  demand  that  its  gov¬ 
ernment  become  more  efficient,  prompting  Vice  President 
A1  Gore  to  initiate  a  National  Performance  Review  to 
“Make  the  entire  federal  government  both  less  expen¬ 
sive  and  more  efficient,  and  to  change  the  culture  of 
our  national  bureaucracy  away  from  complacency 
and  entitlement  toward  initiative  and  empower¬ 
ment”  (Gore,  1993). 

The  late  Secretary  of  Defense  Les  Aspin 
directed  a  “Bottom-Up  Review”  of  DoD  to 
identify  cost  savings  and  improve  efficiency 
and  effectiveness.  In  his  final  report  Aspin 
said:  “We  must  restructure  our  acquisition 
system  to  compensate  for  the  decline  in  avail¬ 
able  resources  for  defense  investment  and  to 
exploit  technological  advances  in  the  commer¬ 
cial  sector  of  our  economy  more  effectively” 
(Aspin,  1993). 

Studies  of  DoD  acquisition  over  the  past  25  years 
reveal  that  (a)  DoD’s  way  of  doing  business  resulted 
in  programs  that  spanned  11  to  21  years  (DiMascio, 
1993),  and  that  (b)  by  the  time  the  weapon  systems  were 
finally  delivered  the  technology  was  outdated.  Significantly, 
the  lengthy  time  to  develop  weapon  systems  was  also  directly 
linked  to  a  doubling  of  the  costs  originally  planned  (Gansler,  1989). 
Based  on  this  past  performance  one  might  expect  higher  costs  in  the 
future.  Unfortunately,  the  ongoing  process  of  federal  deficit  reduction  rules 
out  increased  military  spending.  DoD  must  learn  not  only  to  maintain  the 
technological  superiority  of  the  American  military,  but  learn  to  do  so  in  less 
time  and  at  less  cost. 
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Assumptions 

Jacques  Gansler  warned  against  DoD’s  continuing  preoccupation  with 
technology  without  consideration  of  cost.  Substitute  schedule  for  cost,  and 
one  could  say  the  same  is  true  for  time.  As  Gansler  writes: 

Until  the  DoD  introduces  affordability  [and  schedule]  con¬ 
straints  into  its  requirements  process  and  shifts  from  a 
design-to-performance  approach  to  more  of  a  design-to- 
cost  [and  design-to-schedule]  approach,  it  will  procure  fewer 
and  fewer  weapon  systems  each  year,  and  eventually  the 
United  States  will  not  have  enough  modern  systems  to  pres¬ 
ent  a  credible  defense  posture.  (Gansler,  1989)  [parenthetical 
material  added  to  original] 

It  should  not  take  21  years  to  develop  and  deliver  a  weapon  system  nor  should 
advanced  technology  cost  as  much  as  it  does.  Gansler  points  out  that  perfor¬ 
mance  has  improved  in  commercial  as  well  as  the  defense  industry  because 
of  technology,  “...however,  in  the  defense  world  costs  have  risen  along  with 
performance.”  Comparatively,  “...commercial  computers,  televisions,  and 
other  items  that  use  similar  technology  have  improved  dramatically  in  per¬ 
formance  and  gone  down  dramatically  in  price”  (Gansler,  1989)  and  don’t 
take  as  long  to  produce. 

Methodology 

This  paper  is  a  comparative  analysis  of  the  way  Boeing  and  DoD  used 
technology.  The  problem  was  to  determine  whether  a  difference  in  DoD’s 
approach  to  technology  contributed  to  the  length  of  time  it  took  to  develop 
the  C-17.  This  study  is  based  on  written  works  (published  and  unpublished), 
interviews,  and  observances. 

Research  for  this  report  was  primarily  focused  on  the  DoD  C-17  and  the 
Boeing  777.  It  included  an  extensive  review  of  literature  and  interviews.  The 
literature  review  encompassed  studies,  laws,  standards,  and  articles  relat¬ 
ing  to  various  approaches  to  technology,  their  focuses  and  parameters.  The 
interviews  were  conducted  with  individuals  who  were  or  had  been  involved 
with  the  Boeing  777  or  the  Office  of  Secretary  of  Defense  (OSD).  Additional 
conversations  with  senior  leaders  at  Boeing,  the  Air  Force,  and  DoD  revealed 
their  approaches  to  technology  use  and  their  perceptions. 
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The  Boeing  Approach 

The  777 causes  me  to  sit  bolt  upright  in  bed  periodically.  It’s 
a  hell  of  a  gamble.  There’s  a  big  risk  in  doing  things  totally 
different.  (Dean  Thornton,  President  of  Boeing  Commercial 
Airplane  Group,  Main  1992) 

Boeing  professed  a  belief  that  one  must  approach  technology  with 
an  eye  toward  utility... it  must  earn  its  way  on.  (Condit,  1994) 

Boeing’s  conservative  approach  was  illustrated  in  the  1970s  and  1980s 
when  it  decided  not  to  include  in  its  767  more  advanced  systems  such  as 
fly-by-wire,  fly-by-light,  flat  panel  video  displays,  and  advanced  propulsion 
systems  (Holtby,  1986).  Even  though  the  technology  existed,  Boeing  did  not 
believe  it  was  mature  enough  for  the  767.  Boeing  also  used  what  Gansler 
defines  as  a  design-to-cost  constraint.  After  Boeing  defines  a  program  it 
evaluates  cost  before  going  into  production.  Its  cost  evaluations  include 
trade-offs  of  performance,  technology,  and  manufacturing  investments 
(Boeing,  n.d.). 


In  the  1990s  Boeing  included  in  its  777  (a)  fly-by-wire,  (b)  advanced  liq¬ 
uid-crystal  flat-panel  displays,  (c)  the  company’s  own  patented  two-way 
digital  data  bus  (ARINC  629),  (d)  a  new  wing  the  company  advertised  as  the 
most  aerodynamically  efficient  airfoil  developed  for  subsonic  commercial 
aviation,  (e)  the  largest  and  most  powerful  engines  ever  used  on  a  commer¬ 
cial  airliner,  (f)  nine  percent  composite  materials  in  the  airframe,  and  (g) 
an  advanced  composite  empennage  (Mulally,  1994).  Boeing  also  invested  in 
new  facilities  to  test  the  777  avionics  (Proctor,  1994),  and  to  manufacture 
the  composite  empennage  (Benson,  1995).  Did  Boeing  push  the  technology 
envelope  for  the  777?  Philip  Condit,  Boeing  president,  said  those  were  tech¬ 
nology  improvements,  not  technology  breakthroughs.  He  used  fly-by-wire 
technology  to  illustrate: 
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Fly-by-wire  is  interesting  and  you  can  isolate  it.  But  if  you 
step  back,  our  autopilots  are  fly-by-wire  and  always  have 
been.  We’ve  given  it  a  little  bit  more  authority  [in  the  777].  The 
737  right  from  the  start  had  what  we  called  a  stick  steering 
mode  in  which  you  moved  the  control  wheel  to  make  inputs 
to  the  autopilot.  Fly-by-wire.  The  757 Pratt  Whitney  engine 
was  completely  electronically  controlled. . .  it  makes  neat  writ¬ 
ing,  but  it’s  not  an  order  of  magnitude  change.  Designing  the 
airplane  with  no  mock-up  and  doing  it  all  on  computer  was 
an  order  of  magnitude  change.  (Condit,  1994) 

One  only  has  to  review  the  history  of  airplane  technology  during  the  1980s 
to  see  that  Condit  is  right.  Airbus  and  McDonnell  Douglas  included  fly- 
by-wire  on  the  A340  (Nelson,  1994)  and  the  C-17,  respectively,  during  the 
1980s,  and  both  experienced  problems.  Boeing  was  able  to  learn  from  the 
mistakes  of  Airbus  and  McDonnell  Douglas  (Woolsey,  1994),  and  it  had 
the  advantage  of  using  new  high-powered  ultrafast  computer  chips  that 
increased  throughput.  In  fact  Honeywell,  the  company  that  McDonnell 
Douglas  dismissed  because  it  couldn’t  produce  the  fly-by-wire  fast  enough 
for  the  C-17,  was  the  company  that  successfully  installed  it  on  the  777 
(Woolsey,  1994)— but  not  without  problems. 

Boeing  could  not  assemble  and  integrate  the  fly-by-wire  system  until  it 
solved  problems  with  the  ARINC  693  databus,  the  AIMS-driven  Flight 
Management  System,  and  the  software  coding.  Solving  these  problems  took 
more  than  a  year  longer  than  Boeing  anticipated.  In  order  to  maintain  its 
schedule,  Boeing  did  as  much  as  it  could  without  the  complete  system,  then 
it  used  red-label1  systems  during  flight  tests.  Finally,  the  Federal  Aviation 
Administration  (FAA)  certified  the  last  link,  the  primary  flight  computer,  in 
March,  1995.  In  April,  1995  the  FAA  certified  the  777  as  safe  (Acohido,  1995). 

Technical  Problems 

While  Boeing  may  not  define  its  777  avionics  problems  as  pushing 
the  technology  envelope,  Boeing  did  push  the  envelope  on  its  design  and 
manufacturing  process,  and  its  propulsion.  As  Condit  said,  “Designing 
the  airplane  with  no  mock-up  and  doing  it  all  on  computer  was  an  order 
of  magnitude  change.”  When  one  is  the  first  to  use  a  technology  in  a  new 
way,  one  can  expect  problems.  Assuming  that  Boeing  is  conservative  in  its 
approach,  one  must  ask  why  Boeing  went  from  computer  design  to  build 
with  no  mock-up,  and  why  it  used  new,  large,  high-performance  engines. 


'A  red-label  system  signifies  that  the  system  is  still  in  the  development  and  testing 
phase.  A  black-label  system  signifies  that  hardware  and  software  are  finished  and 
ready  for  production. 
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Computer  and  Aircraft  Design 

Computer-assisted  three-dimensional  interactive  application  (CATIA) 
is  the  computer  application  that  Boeing  used  to  design  the  777  and  improve 
its  manufacturing  process  (Benson,  1994).  Jeremy  Main  best  described  the 
reasons  Boeing  changed  its  way  of  design  and  manufacture  using  CATIA  in 
his  article,  Betting  on  the  21st  Century  Jet. 

As  a  designer,  Boeing  is  preeminent...  I  have  great  respect  for 
them,  hut  they  have  a  long  way  to  go  in  manufacturing. 
Therefore,  to  stay  on  top,  Boeing  must  find  ways  of  building 
planes  better.  If  Boeing’s  new  approach  to  design  works,  the 
777  will  be  an  efficient,  economic  plane  with  a  lot  fewer  bugs 
than  new  planes  usually  have.  As  a  result,  Boeing  could  save 
the  millions  it  usually  spends  fixing  design  problems  during 
production  and  after  the  plane  has  been  delivered  to  the  air¬ 
lines.  (Main,  1992) 


Typically,  engineers  were  still  designing  when  man¬ 
ufacturing  began,  and  they  kept  making  changes  as 
problems  subsequently  came  to  light  on  the  factory 
floor,  on  the  flight  line,  and  even  in  the  customer’s 
hands  after  the  plane  was  delivered. 

Boeing’s  decision  to  use  CATIA  in  conjunction  with  a  team  concept  emerged 
primarily  as  a  means  of  cutting  costs  after  analysis  revealed  that  the  pre¬ 
dominant  cost  drivers  were  rework  on  the  factory  floor  and  downstream 
changes.  The  teams  that  Boeing  calls  design/build  teams  include  repre¬ 
sentatives  from  nearly  every  Boeing  function  involved  in  producing  the 
transport,  plus  customers  and  suppliers  (O’Lone,  1991). 

Typically,  engineers  were  still  designing  when  manufacturing  began,  and 
they  kept  making  changes  as  problems  subsequently  came  to  light  on  the  fac¬ 
tory  floor,  on  the  flight  line,  and  even  in  the  customer’s  hands  after  the  plane 
was  delivered.  For  example,  when  Boeing  delivered  the  747-400  to  United  in 
1990,  it  had  to  assign  300  engineers  to  get  rid  of  bugs  that  it  hadn’t  spotted 
earlier  (Main,  1992).  United  was  not  happy  with  Boeing’s  late  delivery  of 
the  747,  nor  with  the  additional  costs  the  airline  sustained  in  rescheduling 
flights  and  compensating  unhappy  customers  as  a  result  of  maintenance 
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delays.  Boeing  was  deeply  embarrassed  by  delivery  delays  and  initial  service 
problems  of  its  747  (Proctor,  1994).  After  a  lot  of  research  and  deliberation, 
the  company  decided  to  use  computer  aided  technology  more  extensively 
and  change  its  design  and  manufacturing  approach  in  order  to  improve  its 
service.  Yet,  even  though  CATIA  and  the  team  approach  eventually  proved 
worthwhile,  there  were  problems. 

Boeing  encountered  problems  in  adjusting  to  100  percent  computer-aided 
aircraft  design.  Not  only  was  this  a  technology  change,  it  was  a  cultural 
change.  Condit  (1994)  said  engineers  were  reluctant  to  let  others  see  their 
drawings  before  they  were  100  percent  complete.  Ronald  A.  Ostrowski, 
Director  of  Engineering  for  the  777  Division,  said  one  of  the  initial  chal¬ 
lenges  was  to  convert  people’s  thinking  from  2-D  to3-D.  It  took  more  time 
than  we  thought  it  would.  I  came  from  a  paper  world  and  now,  I  am  managing 
a  digital  program  (Quoted  in  Woolsey,  1994). 

The  software  also  had  problems,  and  development  costs  ballooned  slightly 
over  budget  because  of  CATIA.  Boeing  CEO  Frank  Shrontz  said,  “It  was  not 
as  user  friendly  as  we  originally  thought”  (Woolsey,  1994). 

CATIA  and  design/build  teams  were  new  methods  for  applying  technol¬ 
ogy  that  pushed  the  envelope  and  could  have  impacted  Boeing’s  delivery 
schedule.  Instead  of  allowing  a  possible  schedule  slip  and  late  delivery 
to  its  United  customer,  Boeing  decided  to  apply  more  resources,  spend 
the  extra  money,  overcome  its  problems,  and  deliver  its  777  on  sched¬ 
ule.  While  Boeing  did  not  state  how  much  it  spent,  in  April  1992  Fortune 
Magazine  analysts  identified  $3  billion  (Main,  1992)  set  aside  for  research 
and  development  (R&D)  for  the  777.  In  April  1994,  an  editorial  in  Aviation 
Week  and  Space  Technology  (AW&ST)  (estimated  that  final  R&D  costs  for 
the  777  approached  $5.5  billion.  Based  on  the  analysts’  evaluations  one 
could  conclude  that  actual  R&D  costs  were  approximately  $2  billion  over 
planned  costs.  But,  as  Alan  Mulally,  the  Senior  Vice  President  for  Airplane 
Development  and  Definition  said: 

In  our  business  it’s  very  rare  that  you  can  move  the  endpoint.... 

When  you  make  a  commitment  like  we  made  they  [United] 
lay  out  their  plans  for  a  whole  fleet  of  airplanes  so  it’s  a  big 
deal.  They’ll  have  plans  to  retire  old  airplanes.  We  could  have 
stretched  it  out  but  it  just  seemed  best  to  us  to  keep  the  end 
date  the  same  and  add  some  more  resources.  (Mulally,  1994) 
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The  wisdom  of  Mulally’s  decision  was  proven  a  thousand  times  over.  The 
wing  assembly  tool  built  by  Giddings  &  Lewis  in  Janesville,  Wisconsin,  and 
the  world’s  largest  C-frame  riveting  system  built  by  Brotje  Automation  of 
Germany,  were  both  run  in  Seattle  on  programs  generated  by  the  CATIA 
(Benson,  1995).  Engineers  designed  parts  and  tools  digitally  on  CATIA  to 
verify  assembly  fit.  In  Kansas,  Boeing’s  Wichita  Division  built  the  lower 
lob,  or  belly,  of  the  777’s  nose  section  using  CATIA  and  digital  preassem¬ 
bly.  In  Japan  the  skins  of  the  airframe  were  built  using  CATIA-generated 
programs.  Workers  at  all  plants  marveled  at  the  way  all  the  parts  built 
by  different  people  all  over  the  world  fit  together  with  almost  no  need  for 
rework  (Benson,  1995).  Charlie  Houser,  product  line  manager  at  Wichita, 
said  it  best: 

CATIA  and  digital  preassembly  let  us  find  areas  of  potential 
interference  before  we  started  production.  The  individual 
assemblies  fit  together  extremely  well,  especially  the  passen¬ 
ger  floor.  That  assembly  includes  composite  floor  beams,  and 
it  went  together  smoother  than  any  floor  grid  of  any  size  that 
we’ve  ever  built  in  Wichita.  (Benson,  1995) 

Engines 

Three  top  companies  will  supply  engines  for  the  Boeing  777:  Pratt  & 
Whitney,  General  Electric,  and  Rolls  Royce.  The  aircraft  was  designed  for 
two  engines  that  are  billed  as: 
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The  largest  and  most  powerful  ever  built,  with  the  girth  of  a 
737’s  fuselage  and  a  thrust,  or  propulsive  power,  of  between 
71,000  and  85,000  pounds  compared  with  about  57,000 
pounds  of  the  latest  747  engine.  Key  factors  in  this  perfor¬ 
mance  are  new,  larger-diameter  fans  with  wide-chord  fan 
blade  designs  and  by-pass  ratios  ranging  from  6-to-l  to  as 
high  as  9-to-l.  The  typical  by-pass  ratio  for  today’s  wide- 
bodyjet  engines  is  5-to-l.  Pratt  &  Whitney  is  furnishing  the 
PW4000  series  of  engines,  General  Electric  is  offering  the 
GE90  series  and  Rolls-Royce  is  offering  the  Trent  800 
series  of  engines.  (Donoghue,  1994) 


Boeing’s  success  at  getting  these  three  companies  to  pro¬ 
duce  engines  never  before  produced  represent  a  dramatic 
change  from  the  time  when  the  federal  government 
was  the  leader  in  technology.  For  example  in  the  1960s 
General  Electric  didn’t  want  to  risk  the  cost  and  time 
to  develop  a  high-bypass  jet  engine  for  the  747.  General 
Electric  was  content  to  let  a  military  development  pro¬ 
gram,  the  C-5A,  absorb  the  cost  and  time  associated 
with  enhancing  high-bypass  jet  engine  technology 
(Newhouse,  1982).  For  the  777  Boeing  not  only  pushed 
for  new,  more  powerful  engines,  it  also  pushed  for  early 
approval  from  the  Federal  Aviation  Administration  for 
the  plane  to  fly  over  oceans  (called  ETOPS:  extended-range 
twin- engine  operations)  (Mintz,  1995). 


Normally,  the  FAA  first  certifies  a  twin-engine  plane  for  flights  of  not 
more  than  one  hour  from  an  airport,  then  two  hours,  and  finally,  after  a 
couple  years’  service,  a  full  three  hours  so  the  plane  could  fly  anywhere  in 
the  world.  The  767,  powered  by  Pratt  &  Whitney  JT9D-7R4D/E  turbofan 
engines,  became  the  first  Boeing  twin  to  win  120-minute  approval  in  May, 
1985,  but  not  until  after  it  had  flown  for  two  years  (Woolsey,  1991).  Jerry 
Zanatta(1994,  Director,  777  Flight  Test  Engineering,  pointed  out  that 
engines  are  so  reliable  today,  an  airplane  could  travel  on  only  one  engine. 
Flying  with  two  engines  allows  redundancy  that  a  pilot  wants  in  order  to 
ensure  safety  of  flight.  Flying  with  more  than  two  engines  only  increases 
fuel  cost  and  operating  costs  unnecessarily. 


Why  did  Boeing  push  propulsion  technology?  The  answer  is  competition. 
Boeing’s  customer  airlines  are  concerned  about  operating  costs,  and  a 
two-engine  plane  costs  much  less  to  operate  than  a  three-  or  four-engine 
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plane.  Boeing’s  competition.  Airbus,  has  a  twin-engine  plane  (A330)  (Duffy, 
1994)  that  competes  favorably  with  the  777.  If  Boeing  can’t  deliver,  the 
Airbus  can.  Still,  producing  a  new  engine  was  not  without  its  problems. 
For  example,  the  Pratt  and  Whitney  engine  had  performed  perfectly  in  the 
testing  laboratory;  but  on  its  first  test  flight  in  November,  1993,  it  backfired 
several  times. 


The  engine  backfired  because  of  differences  in  the  rates  of  thermal  expan¬ 
sion  between  the  interior  components  of  the  engine  and  the  compressor 
case.  The  case  expanded  faster  than  actively  cooled  interior  engine 
components,  creating  a  space  between  the  blades  and  the  case. 
After  the  first  flight,  engineers  changed  the  software  com¬ 
mands  that  direct  the  variable  blade  angle  of  the  first  four 
compressor  stages  to  reduce  the  temperature  of  the  air 
inside.  On  the  next  flight  the  engine  worked  perfectly 
(Kandebo,  1993). 

Summary  of  the  Boeing  Experience 

Boeing  looked  at  its  investment  in  the  777  and  its 
manufacturing  process  from  a  tactical  and  strategic 
view.  It  was  committed  to  a  successful  777  that  would 
serve  its  customers  and  protect  its  market  share  against 
competition  for  50  years  into  the  future.  Boeing  was  also 
committed  to  changing  and  improving  its  manufacturing 
process  using  the  power  of  computers  so  it  could  improve 
quality  and  cut  costs  well  into  the  21st  century.  As  a  result 
Boeing  management  and  its  Board  of  Directors  were  focused 
on  what  they  had  to  do  to  make  it  all  happen.  They  were  willing  to 
commit  Boeing  resources  toward  overcoming  potential  challenges  that 
included  computer  and  process  technology. 


When  Boeing  underestimated  the  challenge  of  the  design-build  concept 
using  CATIA,  it  could  have  stretched  the  schedule  to  spread  additional  costs 
over  a  longer  time  period.  But  that  would  have  meant  missing  the  delivery 
date  to  United  for  the  first  777.  Boeing  management  made  a  conscious  deci¬ 
sion  to  continue  and  learn  on  its  first  block  of 777s  so  that  all  future  aircraft 
could  benefit. 


We  could  have  stretched  it  out,  hut  it  just  seemed  best  to  us 
to  keep  the  end  date  the  same  and  add  some  more  resources. 
(Mulally,  1994) 


Defense ARJ,  January  2016,  Vol.  22 No.  1 : 28-55 


39 


A  Publication  of  the  Defense  Acquisition  University 


http://www.dau.mil 


The  DoD  Approach  to  Technology 

Technology  on  the  C-17  was  not  as  well  defined  as  some  would 
havens  believe.  (Brig.  Gen.  RonKadish,  1994) 

I  was  shocked  in  the  Fall  of 1992  to  discover  that  this  airplane 
was  being  produced  from  paper,  that  they  did  not  have  a  CAD/ 

CAM  system.  That  they  had  never  had  a  CAD/CAM  system. 

(Gen.  Ronald  Fogleman,  1995) 

Secretary  of  Defense  Harold  Brown  justified  using  a  fixed-price  incen¬ 
tive  contract  to  produce  the  C-17  for  two  reasons:  (a)  Congress  and  President 
Carter  wanted  to  eliminate  cost-plus  contracts  in  order  to  reduce  excessive 
overruns  (Hopkins  &  De  Keyrel,  1993),  and  (b)  all  the  technology  for  the 
C-17  was  already  proven.  The  Advanced  Medium  STOL  Transport  (AMST) 
prototypes  proved  short-field  take  off  and  landing  (STOL)  could  work,  and 
all  hardware  and  software  was  off-the-shelf  (Smith,  1993a).  The  Air  Force 
request  for  proposal  stated  that  “Undue  complexity  or  technical  risk  will 
be  regarded  as  poor  design”  (Johnson,  1986).  After  McDonnell  Douglas 
won  the  competition,  this  theme  was  carried  over  into  the  C-17  technical 
planning  guide: 

The  C-17’s  systems  are  straightforward  in  design,  are  highly 
reliable,  and  represent  current  technology.  For  example, 
a  version  of  the  C-17’s  engine  has  been  proven  in  commer¬ 
cial  airline  service  since  1985.  New  technology  systems, 
like  the  onboard  inert  gas  generating  system  (OBIGGS), 
are  used  only  where  they  offer  significant  advantages  over 
previous  methods....  Avionics  and  flight  controls  that  include 
computer-controlled  multifunction  displays  and  head-up 
displays  enable  the  aircraft  to  be  flown  and  all  its  missions 
accomplished  with  a  flight  crew  of  only  two  pilots  and  one 
loadmaster.  (McDonnell  Douglas,  1993) 

However,  the  C-17  experience  revealed  what  studies  conducted  during  the 
AMST  had  proven  and  Kadish  had  pointed  out— “the  technology  was  not 
as  well  defined  as  some  would  lead  us  to  believe.”  Although  McDonnell 
Douglas  did  not  develop  new  technologies  for  the  C-17,  the  way  in  which 
the  technologies  were  used  was  new.  The  C-17  was  a  new  cargo  airlifter 
dependent  on  a  complex  integrated  avionics  system  to  reduce  the  aircrew 
size  to  two  pilots  and  a  cargo  loadmaster.  By  comparison  the  C-141  and  the 
C-5  use  two  pilots,  a  navigator  for  tactical  and  airdrop  missions  (C-141  only). 
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two  flight  engineers,  and  two  cargo  loadmasters  when  carrying  passengers 
(Lossi,  1995;  Moen,  1995).  Also,  using  STOL  capability  on  a  plane  expected 
to  fly  2,400  nautical  miles  (NM)  with  a  172,200 -pound  payload  including 
outsized  cargo  was  much  different  than  using  STOL  on  a  plane  expected  to 
fly  a  400-mile  radius  with  a  27,000-pound  payload.  The  plane  would  require 
anew  wing  and,  as  John  Newhouse  (1982)  points  out  in  his  book,  The  Sporty 
Game,  “There  is  more  technology  in  the  wing  than  in  any  other  part  of  an 
airframe. ..production  schedules  are  keyed  to  wings.”  The  differences  in 
design  between  a  tactical  STOL  and  a  strategic  STOL  were  the  catalysts 
that  caused  schedule  slips  and  cost  money. 


Advanced  Medium  STOL  Transport 

The  AMST  was  the  genesis  for  the  C-17.  In  1971  the  Air  Force  contracted 
both  Boeing  and  McDonnell  Douglas  to  build  a  prototype  that,  in  the  words 
of  Gen.  Carlton,  was  “really  a  miniature  C-5”  (Kennedy,  n.d.)  to  transport 
cargo  in-theater.  The  plane  was  to  fly  a  400  NM  radius  mission,  carry  27,000 
pounds,  and  land  on  short  runways  using  short  landing  and  take-off  (STOL) 
technology.  McDonnell  Douglas’  YC-15  and  Boeing’s  YC-14  prototypes  suc¬ 
cessfully  demonstrated  powered  lift  technology  in  1975  that  met  mission 
requirements  (Kennedy,  n.d.).  In  March  1976,  the  Air  Force  Chief  of  Staff, 
Gen.  David  C.  Jones,  asked  Air  Force  Systems  Command  to  see  if  it  was  pos¬ 
sible  to  use  a  single  model  of  the  AMST  for  both  strategic  and  tactical  airlift 
roles,  and  if  it  was  possible  to  develop  non-STOL  derivatives  of  the  AMST 
prototype  to  meet  strategic  airlift  missions  (Jones,  1976).  It  appears  that 
this  strategic  study  originated  with  a  note  from  the  Chairman  of  the  Joint 
Chiefs  of  Staff,  Gen.  George  S.  Brown,  that  asked  “Is  it  practical  to  have  an 
AMST  with  a  slightly  higher  box  pick  up  much  of  the  C-5  outsized  load  for 
Europe— with  air  refueling  as  necessary?”  (Lemaster,  1976). 
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Gordon  Taylor  and  Gordon  Quinn  from  the  Aeronautical  Systems  Division 
at  Wright  Patterson  Air  Force  Base,  Ohio,  were  leaders  in  a  conceptual 
design  analysis  to  determine  if  DoD  could  use  the  AMST  for  strategic 
missions.  The  analysis  included  reviewing  the  ability  to  carry  the  M-60 
Main  Battle  tank,  weighing  110,000  to  117,000  pounds,  on  a  routine  basis 
with  ranges  from  2,000  NM,  3,000  NM,  and  4,000  NM.  Taylor  and  Quinn 
concluded  that  using  a  derivative  aircraft  in  a  routine  strategic  airlift  role 
would  increase  AMST  weight  and  cost  significantly.  To  restructure  the 
AMST  from  a  tactical  to  a  strategic  program  would  require  full-scale  devel¬ 
opment  (a  larger  wing,  heavier  structure,  and  different  aerodynamics).  Even 
in  a  non-STOL  capacity  the  wing  was  the  major  airframe  component  that 
the  study  said  must  undergo  considerable  change  (Taylor  &  Quinn,  1976). 
In  May  1976,  Brig.  Gen.  Philip  Larsen,  Deputy  Chief  of  Staff,  Systems,  Air 
Force  Systems  Command,  wrote: 

It  would  not  be  cost  effective  to  incorporate  a  STOL  capa¬ 
bility  in  a  strategic  airlift  derivative  aircraft.  A  strategic 
derivative  could  employ  a  less  complex  conventional  flap 
system  which  would  permit  CTOL  [conventional  takeoff  and 
landing]  operations  from  an  8,000 foot  hard  surface  runway 
under  sea  level  standard  day  conditions.  The  aircraft  would 
be  stretched  eight  feet  to  provide  a  55-foot-long  cargo  com¬ 
partment.  This  would  permit  routinely  carrying  the  M-60 
tank  and  single  item  payloads  up  to  112,500  pounds,  or  14 
463L  cargo  pallets,  for  distances  up  to  3,000  NM  without 
refueling.  In  this  particular  example,  it  would  be  necessary 
to  increase.  ..YC-15  wing  area  69  percent  and  gross  weight  115 
percent.  (Larsen,  1976) 

On  December  10, 1979,  Program  Management  Directive  (PMD)  No.  R-Q 
6131(3)  formally  cancelled  the  AMST  program.  On  that  same  day  PMD  No. 
R-C  0020(1)  provided  formal  direction  and  guidance  for  activities  leadingto 
Full  Scale  Engineering  Development  of  the  C-X.  PMD  R-C  0020(1)  directed 
that  the  C-X  skip  Milestone  I  and  the  Demonstration  and  Validation  phase 
because  “...the  new  aircraft  will  use  existing  technology...  since  the  Air 
Force  had  demonstrated  and  proved  advanced  technology  concepts  and 
operational  utility  in  the  AMST  program”  (Johnson,  1986). 

Changing  Payload  Requirements 

Payload  requirements  changed  at  least  five  times  over  the  life  of  the 
C-17.  Beginning  in  1981  the  request  for  purchase  asked  for  a  STOL  plane  that 
could  carry  a  payload  of  130,000  pounds  (Air  Mobility  Command  [AMC], 
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1993).  McDonnell  Douglas  claimed  it  could  produce  a  STOL  plane  that  could 
carry  172,200  pounds  2,400  miles  (Johnson,  1986).  When  the  contract 
was  awarded  in  1982,  the  payload  requirements  were  changed  to  172,200 
pounds  (AMC,  1993).  DoD  did  not  evaluate  the  cost  to  grow  from  a  payload 
of  130,000  pounds  to  172,200  pounds.  In  1988  DoD  changed  the  payload 
requirement  from  172,200  pounds  to  167,000  in  order  to  accommodate  the 
addition  of  a  4-pallet  ramp  and  OBIGGS  that  added  5,000  pounds  additional 
weight  to  the  aircraft  (Snider,  1992).  In  1991  Gen.  Hansford  Johnson,  MAC 
Commander,  reduced  the  payload  requirements  from  167,000  pounds  to 
160,000  pounds  because  the  kinds  of  equipment  MAC  needed  to  haul  over 
essential  routes— from  West  Coast  bases  to  Hickam  AFB,  Hawaii,  and  from 
East  Coast  bases  to  Lajes  airfield  in  the  Azores— did  not  require  a  plane  with 
a  167,000-pound  capacity.  He  said: 

This  was  not  a  reassessment  of  requirements  as  much  as  it 
was  a  refinement  of  the  original  requirements...  McDonnell 
Douglas,  in  competing  for  the  contract,  offered  more  than 
what  MAC  needed....  All  of  us,  being  eager  to  do  more,  said 
sure,  we’ll  write  the  specs  at  the  higher  level.  (Morrocco,  1991) 


Payload  requirements  changed  at  least  five  times 
over  the  life  of  the  C-17. 


In  January  1995,  DoD,  Congress,  and  McDonnell  Douglas  agreed  to  decrease 
the  payload  requirement  even  more.  If  the  C-17  were  to  carry  a  160,000- 
pound  payload  using  STOL  capability  with  the  weight  of  the  plane  and  the 
required  fuel,  it  needed  more  powerful  engines.  Pratt  &  Whitney  and  Rolls 
Royce  had  produced  more  powerful  engines,  but  the  Under  Secretary  of 
Defense  for  Acquisition,  John  M.  Deutch,  said  changing  to  more  powerful 
engines  was  too  costly.  He  preferred  to  reduce  payload  specifications  rather 
than  change  engines,  especially  since  the  C-17  did  not  need  to  carry  a  greater 
payload  to  perform  its  mission  (Morrocco,  1994).  Fogleman  said  that  DoD 
“Allowed  the  plane  to  be  over  spec’d  unnecessarily....  We  didn’t  need  a  plane 
to  carry  a  172,200-pound  payload  then  and  we  don’t  need  a  plane  to  carry 
160,000  pounds  now”  (Fogleman,  1995). 


Defense ARJ,  January  2016,  Vol.  22 No.  1 : 28-55 


43 


A  Publication  of  the  Defense  Acquisition  University 


http://www.dau.mil 


An  absolute  critical  leg  for  us  in  this  new  world  we  are  liv¬ 
ing  in  is  how  much  can  this  airplane  carry  3,200  miles... 
we  established  a  110,000-pound  payload  threshold  at  the 
3, 200-mile  range...  The  original  requirement  set  in  the  early 
1980s  was  for  a  130, 000 -pound  payload,  the  weight  of  an  M-l 
tank  then...  this  specification  is  now  not  considered  the  most 
critical.  It  was  linked  to  the  Cold  War  goal  of  transporting  10 
Army  divisions  to  Europe  in  10  days,  rather  than  how  to  deal 
with  the  types  of  regional  contingencies  the  Pentagon  now  is 
focusing  on  in  its  planning.  An  absolute  critical  leg  for  us  in 
this  new  world  we  are  living  in  is  how  much  can  this  airplane 
carry  3,200  miles....  So  we  established  a  110,000-pound  pay- 
load  threshold  at  the  3,200-mile  range  which  did  not  exist 
before... the  aircraft  meets  that  goal  and  is  projected  to  exceed 
it.  Sticking  to  the  original  specification  would  have  required 
switching  to  more  powerful  engines.  (Morrocco,  1994) 

On  January  17, 1995,  the  Air  Mobility  Commander,  Gen.  Robert  Rutherford, 
declared  the  C-17  a  success  when  he  certified  it  operationally  capable 
(McDonnell  Douglas,  1995).  It’s  worth  noting,  however,  that  the  program 
did  not  begin  to  overcome  technology  problems  until  after  top-level  com¬ 
mitment  was  apparent  from  principals  like  Deutch  (Defense  Week,  1995) 
and  Fogleman.  Fogleman  essentially  said  this  is  nonsense,  “We  don’t  need 
that  much  payload  capability”  (Fogleman,  1995),  and  Deutch  arranged  a  set¬ 
tlement  with  McDonnell  Douglas  that  allowed  performance  trade-offs  and 
help  with  computer  (CAD/CAM)  technology.  McDonnell  Douglas,  in  turn, 
put  their  best  people  on  the  job  to  produce  a  technically  proficient  airplane 
(Morrocco,  1994).  As  a  result  of  technology  trade-offs  and  top  management 
commitment  from  both  DoD  and  the  contractor,  the  C-17  exceeded  its  sched¬ 
ule  during  1994  and  met  mission  requirements  in  1995. 

Technical  Problems 

One  might  say  that  design  problems  and  planning  problems  were  at  the 
root  of  technical  problems  that  added  time  to  development  of  the  C-17.  The 
underlying  problem  was  that  the  players  underestimated  the  technical  chal¬ 
lenges.  Roger  A.  Panton,  Chief  of  Engineering  at  the  C-17  System  Program 
Office  at  Wright  Patterson  AFB,  said  “Our  primary  technical  problem  with 
the  C-17  was  integration.  We  grabbed  too  much  off  the  shelf  and  tried  to  put 
it  together”  (Panton,  1994).  Critical  off-the-shelf  technology  included  fly-by- 
wire,  advanced  materials,  engines,  software,  and  the  powered  lift  that  the 
McDonnell  Douglas  YC-15  prototype  demonstrated  in  1975. 


44 


Defense  ARJ,  January  2016,  Vol.  23  No.  1 : 28-55 


January  2016 


The  Defense  Science  Board  added  in  a  December  1993  report  that  lack  of 
computer-aided  design  and  engineering  changes  contributed  to  production 
delays  (Defense  Science  Board,  1993).  Deutch  summarized  some  of  the  most 
glaring  weaknesses  (a)  technical  risks  involved  in  flight  test  software  and 
avionics  integration;  (b)  structural  deficiencies  in  the  wings,  flaps  and  slats; 
and  (c)  uncertainty  of  flight  test  program  requirements  (Morrocco,  1993). 

Avionics  Integration 

Avionics  is  a  term  that  covers  the  myriad  of  ultrarefined 
electronic  devices  on  which  modern  airplanes  rely. 
(Newhouse,  1982) 

On  the  C-17  that  includes  the  flight  control  system  and  the  mission 
computer.  Integration  of  the  mission  computer  and  electronic  flight  control 
system  was  one  of  the  three  critical  paths  leading  to  first  flight  (Smith,  1990). 
The  first  test  flight  of  the  C- 17,  September  15,  1991,  was  behind  schedule 
(Smith,  1991)  because  of  problems  that  included  changing  from  a  standard 
mechanical  flight  control  system  to  a  quadruple  redundant  electronic  flight 
control  system,  and  delays  in  the  mission  computer  software  and  flight 
control  software  (Hopkins  &  De  Keyrel,  1993). 
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In  1987,  after  McDonnell  Douglas  missed  delivery  of  the  first  test  aircraft, 
DoD  reduced  funding  during  budget  reductions  and  moved  delivery  sched¬ 
ule  for  the  first  test  aircraft  three  years  to  the  right  (to  July,  1990)  (Mastin, 
1994).  In  addition,  in  January  1988,  Congress  deducted  $20  million  from  the 
C-17  during  its  budget  review,  but  invited  DoD  to  ask  for  reprogramming  of 
funds  (SAF/AQ,  1989).  DoD  declined. 

Flight  Control  System 

McDonnell  Douglas  changed  to  an  electronic  flight-control  system  to 
prevent  the  plane  from  entering  into  a  deep  stall  (Hopkins  &  De  Keyrel, 
1993).  Wind  tunnel  testing  revealed  that  the  C-17  design  caused  deep  stall 
characteristics.  In  1987  the  Sperry  Corporation  (the  flight- control  subcon¬ 
tractor)  told  McDonnell  Douglas  that  the  mechanical  flight  control  system 
could  not  prevent  pilots  from  putting  the  airplane  into  an  irreversible  stall 
(ASD/AF/C-17, 1987).  After  confirming  that  the  aircraft  configuration  and 
the  mechanical  flight  control  system  could  allow  the  aircraft  to  enter  an 
uncontrollable  stall  during  certain  tactical  maneuvers,  Douglas  directed 
Sperry  to  change  the  mechanical  flight  control  to  a  fly-by-wire  system 
(Smith,  1993).  During  this  same  period  Honeywell,  Incorporated,  purchased 
the  Sperry  Corporation. 

In  June  1989,  Honeywell  officials  established,  April  25,  1991,  as  the  new 
delivery  date  for  flight  qualified  software.  The  additional  delay  added  four 
years  from  the  time  Douglas  first  asked  for  the  system  change  until  delivery 
(1987-1991).  Even  though  Honeywell  successfully  completed  an  interface 
control  document  (ICD)  in  July  1989,  showing  how  the  electronic  flight  con¬ 
trol  system  (EFCS)  interacted  with  subsystems,  the  additional  delay  was 
too  much.  Brig.  Gen.  Michael  Butchko,  Air  Force  C-17  program  manager, 
convinced  Douglas  Aircraft  to  hire  General  Electric  (GE)  for  development 
of  a  similar  system  as  a  precautionary  measure  (Hopkins  &  De  Keyrel,  1993). 
Douglas  ended  Honeywell’s  contract  for  the  EFCS  in  July  1989  (Thomas 
et  al.,  1990).  GE  delivered  the  version  1  software  for  integration  testing  in 
October  1990  (Thompson,  1991). 

Mission  Control  Computer 

The  three  mission  computers  receive  data  from  other  systems,  ana¬ 
lyze  data,  perform  calculations,  and  display  information  to  the  pilot  and 
copilot.  The  computers  act  as  the  heart  of  the  automated  avionics  system 
and  perform  functions  normally  done  by  the  flight  engineer  such  as  deter¬ 
mining  an  estimate  of  position  and  velocity,  weight  limits,  airdrop,  small 
airfield  approaches,  and  system  management  (Thomas  et  al.,  1990).  Each 
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mission  computer  performs  its  calculations  and  then  compares  its  results 
with  the  solutions  broadcast  over  the  data  bus  by  the  other  two  computers 
(McDonnell  Douglas,  1993). 

Douglas  awarded  a  firm-fixed-price  contract  to  Delco  in  July,  1986,  to 
develop  the  mission  computer  (Mundell,  1990).  In  August  1988,  an  indepen¬ 
dent  review  team  that  included  personnel  from  McDonnell  Douglas,  Hughes 
Electronics,  and  the  Air  Force  concluded  that  Delco  had  not  adequately 
accomplished  system  engineering  and  that  McDonnell  Douglas  had  not 
adequately  defined  the  mission  computer  system  requirements.  Delco  devel¬ 
oped  the  mission  computer  software  enough  to  hold  a  critical  design  review 
of  the  detail  design  in  Apr  il,  1989  for  the  first  of  two  increments  of  software, 
but  it  would  not  commit  to  a  plan  for  completing  the  mission  computer.  In 
July  1989,  Douglas  and  Delco  signed  an  agreement  that  partially  termi¬ 
nated  Delco’s  contract  for  the  mission  computer  subsystem,  and  Douglas 
assumed  responsibility  for  managing  the  overall  software  development 
effort  (Thomas  et  al.,  1990). 


McDonnell  Douglas  subcontracted  a  majority  of  software  for  the  C-17  to 
subcontractors  and  suppliers.  During  this  process  Douglas  did  not  specify 
a  specific  computer  language,  which  resulted  in  software  for  the  C-17  in 
almost  every  known  language  of  the  time  (AW&ST,  1992).  Integration  of  the 
software  was  a  nightmare  that  the  Government  Accounting  Office  (GAO) 
said  resulted  in  “The  most  computerized,  software-intensive  aircraft  ever 
built,  relying  on  19  different  embedded  computers  incorporating  more 
than  80  microprocessors  and  about  1.3  million  lines  of  code”  (Hopkins  & 
De  Keyrel,  1993).  The  final  software  release  was  in  September,  1994,  with 
upgrades  through  March  1995.  David  J.  Lynch,  in  his  article  “Airlift’s  Year 
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of  Decision,”  said  that  in  1994  the  mission  computer  remained  slow  and  did 
not  meet  the  desired  throughput  capacity  requirements  (Lynch,  1994).  John 
Wilson,  C-17  deputy  program  manager,  acknowledged  that  the  program 
office  needs  to  consider  software  improvements: 

This  is  a  tough  area.  The  C-17  System  Program  Office  recog¬ 
nizes  that  additional  throughput  could  be  beneficial.  Although 
the  computer  performs  the  basic  mission,  it  is  slow  and  does 
not  meet  the  desired  throughput  capacity.  We  are  working 
the  area.  (Wilson,  1995) 


Wings 

The  wings,  flaps,  and  slats  combine  with  high  thrust  engines  and  the 
electronic  flight  control  system  for  STOL.  Exhaust  from  the  jet  engines  force 
air  over  wings  and  flaps,  generating  additional  lift.  Engines  on  the  C-17  are 
mounted  under  the  wings  and  large  flaps  protrude  down  into  the  exhaust 
stream.  The  engine  exhaust  is  forced  through  the  flap  and  down  both  sides  of 
the  flap,  creating  significant  added  lift.  The  externally  blown  flap  system  and 
the  full-span  leading  edge  slats  enable  the  C-17  to  operate  at  low  approach 
speeds  for  short-field  landings  and  for  airdrops  (Henderson,  1990).  Powered 
lift  enables  the  C-17  to  land  on  shorter  runways  than  current,  large-capacity 
transports  by  allowing  it  to  fly  slow,  steep  approaches  to  highly  accurate 
touchdown  points  (McDonnell  Douglas,  1993).  In  October  1992,  the  wing 
failed  a  wing-strength  test  (Morrocco,  1993).  Even  though  Air  Force  had 
reduced  the  maximum  payload  requirements  in  December,  1989  from 
167,000  pounds  to  160,000  pounds  at  2,400  NM,  the  wings  were  still  not 
strong  enough  to  handle  a  full  payload  (GAO,  1994)  along  with  the  fuel  and 
structure  weight  at  a  1.5  safety  factor.  Causes  of  the  failure  included  a  com¬ 
putational  error  in  the  initial  design,  optimistic  design  assumptions,  and 
the  method  used  to  determine  compression  stress  (Huston  et  al., 

1993).  The  wing  modifications  covered  a  large  area  because 
McDonnell  Douglas  used  the  erroneous  computation 
throughout  the  wing  structure  (Smith,  1993). 

The  failed  strength  test  was  preceded  by  per¬ 
sistent  fuel  leaks  around  the  wing  in  September 
1991,  because  holes  were  not  drilled  and  fas¬ 
tened  properly.  Douglas  held  up  delivery  of 
Production  Aircraft  for  nearly  a  month  while  tech¬ 
nicians  located  the  leaks.  Jim  Berry,  then  Douglas 
vice-president  and  general  manager  of  the  C-17 
program,  said  the  problems  stemmed  primarily 
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from  a  lack  of  production  discipline  and  unscheduled  work.  The  failed  wing- 
strength  test  and  persistent  fuel  leaks  around  the  wing  cost  McDonnell 
Douglas  more  than  $1  billion,  and  modifications  added  an  additional  700 
pounds  in  aircraft  weight  (Smith,  1993). 

Summary  of  the  DoD  Experience 

DoD  did  not  look  at  its  investment  in  the  C-17  from  a  technically  stra¬ 
tegic  view,  nor  did  it  appreciate  the  challenge  of  C-17  STOL  technology. 
When  DoD  changed  the  mission  of  the  tactical  STOL  to  a  strategic  STOL, 
both  McDonnell  Douglas  and  the  DoD  underestimated  the  scope  and  cost 
of  the  effort  necessary  to  reduce  the  aircrew  size  to  three  persons  and  fly 
2,400  NM  with  a  172,200-pound  payload.  As  Fogleman  said,  DoD  “...allowed 
the  plane  to  be  over  spec’d  unnecessarily....  We  didn’t  need  a  plane  to  carry 
a  172,200-pound  payload  then  and  we  don’t  need  a  plane  to  carry  160,000 
pounds  now”  (Fogleman,  1995).  In  both  cases  (reducing  aircrew  size  and 
requiring  STOL)  McDonnell  Douglas  had  to  increase  its  use  of  computerized 
flight  controls  in  order  to  maximize  performance.  In  all  cases  lack  of  experi¬ 
ence  with  software  caused  schedule  delays  and  increased  cost.  In  addition  a 
math  error  caused  problems  that  prevented  the  C-17  wing  from  passing  the 
stress  test  at  150  percent.  If  McDonnell  Douglas  had  a  CAD/CAM  system 
like  CATIA,  it  might  have  detected  and  prevented  both  the  stress  problems 
and  the  fuel  leak  problems. 


Contrasting  the  DoD 
and  Boeing  Approaches 

Boeing’s  focus  during  the  design  and  acquisition  process  was  on  cost, 
schedule,  performance,  and  market  competition.  DoD’s  focus  during  the 
design  and  acquisition  process  was  on  performance.  Boeing  looked  at 
the  technology  included  in  its  airplane  more  realisti¬ 
cally  and  did  not  try  to  include  more  than  the  market 
would  buy.  DoD,  on  the  other  hand,  gold-plated 
requirements  by  providing  more  capacity  than  the 
customer  needed,  and  underestimated  the  STOL 
technology  and  cost  needed  to  carry  a  172,200- 
pound  payload.  Boeing  used  the  CATIA  computer 
program  to  help  revolutionize  its  design  and  man¬ 
ufacturing  plant  so  that  parts  would  fit  right,  and 
built  an  entirely  new  plant  to  integrate  and  test 
its  new  avionics  package.  Boeing’s  investment  in 
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infrastructure  helped  overcome  its  many  computer  and  avionics  prob¬ 
lems.  DoD’s  contractor,  McDonnell  Douglas,  designed  the  C-17  on  paper. 
McDonnell  Douglas  did  not  use  a  computer  program  that  could  have  iden¬ 
tified  and  helped  eliminate  both  the  wing  stress  and  the  fuel  leak  problems, 
and  it  did  not  adequately  plan  integration  of  the  C-17  avionics  package. 

When  Boeing  underestimated  the  time  and  cost  to  overcome  technical 
problems  in  the  777  fly-by-wire  and  CATIA,  it  determined  what  it  needed 
to  do  to  correct  the  problems.  Boeing  decided  to  meet  its  delivery  date  to 
United,  and  commit  additional  money  and  resources  to  solve  the  technical 
problems.  DoD,  on  the  other  hand,  upon  learning  that  McDonnell  Douglas 
could  not  meet  its  first  scheduled  flight  because  of  technical  problems  that 
included  software  and  STOL  design,  took  money  away  from  the  program 
and  stretched  it  out  three  years. 


What  happens  is  that  DoD  takes  so  long  to  overcome 
technology  problems  that  by  the  time  a  weapon  is 
complete,  the  technology  is  outdated. 


Jacques  Gansler  in  his  book,  Affording  Defense,  explains  how  DoD’s  preoc¬ 
cupation  with  technology  is  self  defeating: 

The  unreasonably  long  acquisition  cycle  (10-15 years)... leads 
to  unnecessary  development  costs,  to  increased  “gold  plat¬ 
ing,”  and  to  the  fielding  of  obsolete  technology.  (Gansler,  1989) 

What  happens  is  that  DoD  takes  so  long  to  overcome  technology  problems 
that  by  the  time  a  weapon  is  complete,  the  technology  is  outdated.  In  the 
case  of  the  C-17,  that’s  true.  It  is  the  most  versatile  up-to-date  cargo  plane 
the  United  States  currently  has,  but  DoD  couldn’t  produce  the  C-17  until  the 
technology  problems  of  design,  fly-by-wire,  embedded  computer  systems, 
and  wing  stress  were  solved.  As  a  result,  Boeing  completed  the  777  at  about 
the  same  time  even  though  it  was  conceived  several  years  after  the  C-17. 
The  777  uses  the  same  level  of  technology  or,  as  with  flat-panel  displays, 
computer-design,  increased  propulsion,  and  manufacturing  processes,  it 
uses  more  advanced  technology. 
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Jacques  Gansler  describes  the  dilemma  between  the  Defense  and  commer¬ 
cial  approach  to  technology  in  his  illustration  of  a  college  student  working 
in  the  commercial  world  versus  one  who  works  for  defense. 

A  typical  American  engineering  student  (graduate  or 
undergraduate)  is  taught  how  to  design  the  “best  system.” 

Using  computers,  sophisticated  mathematics,  and  all  their 
engineering  skills,  these  students  set  out  to  design  systems 
that  will  achieve  the  maximum  performance.  If  they  enter 
the  commercial  world,  they  are  taught  that  their  designs 
should  be  modified  to  reduce  the  likely  costs  of  production 
and  operation.  However,  if  they  enter  the  defense  world,  they 
continue  to  use  the  design  practices  they  learned  in  school, 
and  cost-cutting  becomes  an  exercise  for  the  manufacturer. 
(Gansler,  1989) 

If  DoD  continues  its  past  preoccupation  with  technology,  it  will  fall  behind. 
In  the  past,  commercial  development  programs  leveraged  the  technology 
developed  by  the  military;  this  was  certainly  true  for  the  777  fly-by-wire. 
However,  the  military  is  now  learning  from  commercial  developers.  The 
F-22  and  other  acquisition  programs  are  using  the  integrated  product  teams 
that  Boeing  developed  in  its  design-build  approach.  The  F-22,  the  B-2,  and 
the  V-22  Osprey  are  all  benefitting  from  CATIA  and  the  strides  Boeing  made 
in  composite  manufacturing.  However,  the  programs  are  not  benefitting 
from  Boeing’s  design-to-cost  approach. 


Conclusions 

Did  the  difference  in  approaches  to  technology  contribute  to  the  length 
of  time  it  took  to  develop  the  DoD  C-17  compared  to  the  Boeing  777?  One 
would  have  to  say  yes.  The  most  telling  difference  was  how  Boeing  and  DoD 
reacted  to  technical  problems  that  threatened  to  impact  delivery  dates. 
Boeing  added  more  resources  to  overcome  technical  problems  whereas 
DoD  took  resources  away  and  moved  the  delivery  date  out  three  years.  As 
long  as  DoD  overestimates  the  maturity  of  technology  it  wants  to  use,  asks 
for  more  technology  than  it  needs,  does  not  commit  resources  to  overcome 
technology  problems  in  a  timely  manner,  and  does  not  require  cost,  schedule, 
and  technology  trade-offs  during  evolution  of  the  design,  it  will  take  longer 
to  develop  weapon  systems. 
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This  article  uses  transaction  costs  to  predict  the  probability  of  incurring  a 
cost  breach  in  a  major  defense  acquisition  program  (MDAP).  As  transaction 
costs  are  not  explicitly  measured  for  MDAPs,  the  authors  use  estimates  of 
systems  engineering  and  program  management  (SE/PM)  costs  as  a  share 
of  overall  program  costs  as  a  proxy  for  transaction  costs.  Using  survival 
analysis,  a  new  approach  to  predicting  cost  breaches,  they  also  found  that 
an  increased  share  of  SE/PM  costs  in  initial  program  estimates  can  help 
predict  future  cost  breaches. 
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Controlling  cost  growth  for  a  major  defense  acquisition  program 
(MDAP)  has  been  problematic  in  the  Department  of  Defense  (DoD) 
for  many  years.  A  2007  RAND  study  of  cost  growth  in  DoD  weapon 
systems  determined  that  the  cost  of  the  46  programs  studied  was 
more  than  1.46  times  the  cost  estimate  for  Milestone  B  (program 
initiation)  (Younossi  et  al.,  2007).  According  to  the  Government 
Accountability  Office,  active  MDAPs  in  Fiscal  Year  (FY)  2011 
collectively  experienced  a  cost  growth  of  $74.4  billion  (Sullivan, 
2011). 


The  Selected  Acquisition  Report  (SAR)  was  introduced 
in  1967  to  provide  DoD  and  the  Congress  a  summary 
of  each  MDAP’s  ability  to  meet  cost,  perfor¬ 
mance,  and  schedule  objectives  agreed  upon 
by  the  program  manager  and  defense  acqui¬ 
sition  executive.  Program  managers  were 
now  required  to  provide  a  brief  explanation 
in  the  SAR  of  how  and  why  any  cost  breaches 
occurred. 

Based  on  evidence  that  this  was  insufficient  to 
control  cost  growth,  in  1982  Senator  Samuel 
Nunn  and  Congressman  David  McCurdy 
introduced  the  Nunn-McCurdy  Act  (1983)  to 
hold  DoD  accountable  to  Congress  for  man¬ 
agement  of  program  costs.  The  Nunn-McCurdy 
Act  became  law  with  the  FY  1983  Department 
of  Defense  Authorization  Act,  establishing  congressio¬ 
nal  oversight  for  MDAPs  that  exceed  established  cost 
thresholds.  The  Nunn-McCurdy  Act  has  been  statuto¬ 
rily  amended  a  number  of  times  over  the  years.  One  of 
the  most  significant  changes  to  the  reporting  require¬ 
ments  occurred  in  the  FY  2006  National  Defense 
Authorization  Act  (Pub.  L.  109-163),  when  Congress 
added  the  original  baseline  estimate  as  a  threshold 
against  which  to  measure  cost  growth  (National 
Defense,  2006).  The  new  standard  prevents  DoD 
from  avoiding  a  Nunn-McCurdy  breach  by 
simply  rebaselining  a  program. 
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Clearly,  the  ability  to  anticipate  cost  overruns  before  breaches  occur  would 
be  extremely  valuable  to  program  managers  and  policy  makers.  However, 
the  conventional  focus  on  MDAP  production  costs  potentially  misses  a 
critical  clue.  Angelis,  Dillard,  Franck,  and  Melese  (2008)  hypothesize  that 
the  higher  the  ratio  of  another  key  set  of  costs,  transaction  costs  (costs 
associated  with  “source  selection,...  contract  negotiation  and  management, 
performance  measuring  and  monitoring,  and  dispute  resolutions”)  relative 
to  production  costs,  the  greater  the  likelihood  of  schedule  and  cost  overruns. 

Higher  transaction  costs  are  typically  experienced  in  programs  that  involve 
greater  asset  specificity,  complexity,  and  imperfect  and  asymmetric  infor¬ 
mation— in  other  words,  programs  that  are  at  greater  risk  (Franck  &  Melese, 
2008).  Proxy  measures  first  suggested  by  Angelis  et  al.  (2008)  that  can  be 
used  to  capture  many  of  these  costs  include  systems  engineering  and  pro¬ 
gram  management  (SE/PM)  costs  regularly  reported  by  MDAP  contractors. 
It  seems  reasonable  to  assume  that  combined  SE/PM  costs  will  be  a  higher 
share  of  total  overall  estimated  costs  (production  +  transaction  costs,  as 
predicted  at  the  Milestone  B  decision  point).  Program  managers,  therefore, 
could  reasonably  anticipate  higher  costs  in  coping  with  more  complex  and 
riskier  MDAP  projects.  Unclear  is  whether  or  not  these  costs  are  explicitly 
considered  in  program  cost  estimates  or  simply  reflected  in  the  size  of  SE/ 
PM  staff  assigned  to  more  complex  or  high-risk  programs. 


Background 

Many  studies  (e.g.,  Bolten,  Leonard,  Arena,  Younossi,  &  Sollinger,  2008) 
have  examined  cost  growth  in  DoD  programs,  yet  little  research  has  been 
done  on  the  relationship  between  transaction  costs  and  cost  overruns  as 
suggested  by  Angelis  et  al.  in  2008.  A  2006  RAND  study  established  that 
MDAP  SE/PM  costs  vary  between  programs  depending  on  the  program  type 
(Stem,  Boito,  &  Younossi,  2006),  and  Angelis  et  al.  (2008)  suggested  using 
the  SE/PM  cost  as  a  proxy  for  transaction  costs  to  examine  the  relationship 
between  transaction  costs  and  cost  overruns. 

In  general,  a  program  has  two  types  of  costs:  production  costs  and  transac¬ 
tion  costs.  Production  costs  are  usually  captured  in  the  Work  Breakdown 
Structure  (WBS),  but  transaction  costs  may  not  be  adequately  captured  in 
the  WBS.  Because  traditional  cost  estimates  are  based  on  the  production 
costs  found  in  the  WBS,  they  do  not  explicitly  include  transaction  costs 
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(Angelis  et  al.,  2008).  Although  they  are  not  often  captured  in  the  account¬ 
ing  records,  the  time  and  effort  associated  with  complex  and  risky  MDAP 
transactions  represent  real  costs  to  the  organization. 

In  previous  research,  Angelis  et  al.  (2008)  examined  how  transaction  costs 
might  be  captured  in  the  cost  estimates  of  DoD  acquisition  programs. 
Angelis  et  al.  identified  a  number  of  issues  with  DoD  program  management 
cost  data  reported  for  major  weapon  systems  and  found  that  they  are  not 
well  suited  for  developing  a  cost  model  that  includes  transaction  cost  vari¬ 
ables.  As  an  alternate  approach,  they  explored  using  contractor  Program 
Management  data  from  cost  data  summary  reports  (DD  Form  1921)  and 
suggested  using  the  SE/PM  category  as  a  proxy  for  transaction  costs.  The 
DoD  (2011a)  defines  systems  engineering  as  “the  technical  and  management 
efforts  of  directing  and  controlling  a  totally  integrated  engineering  effort 
of  a  system  or  program.”  Program  management  is  defined  as  “the  business 
and  administrative  planning,  organizing,  directing,  coordinating,  con¬ 
trolling,  and  approval  actions  designated  to  accomplish  overall  program 
objectives,  which  are  not  associated  with  specific  hardware  elements  and 
are  not  included  in  Systems  Engineering”  (DoD,  2011,  p.  222).  The  ratio  of 
(SE  +  PM)  costs  relative  to  total  program  costs  offers  a  potentially  valuable 
way  to  compare  transaction  costs  across  different  programs. 


Following  the  method  used  by  Biggs  (2013),  this  study  uses  the  SEPM  cost 
ratio  for  a  program  as  shown  in  Equation  1: 


SE/PM  Cost  Ratio  = 


SE  +  PM  Costs 
Total  Cost 


(1) 


The  numerator  of  the  SE/PM  cost  ratio  is  the  sum  of  SE  and  PM  cost  expen¬ 
ditures  and  the  denominator  is  total  program  expenditures  (estimate  at 
completion,  or  EAC).  A  ratio  is  calculated  to  provide  a  perspective  on  the  rel¬ 
ative  magnitude  of  SE/PM  expenditures  as  well  as  to  allow  for  comparison 
across  different  programs.  The  hypothesis  is  that  programs  with  higher  SE/ 
PM  cost  ratios  are  more  likely  to  experience  cost  breaches  than  programs 
with  lower  SE/PM  cost  ratios.  This  is  based  on  the  assumption  that  higher 
SE/PM  cost  ratios  are  related  to  riskier  contractual  relationships  since 
more  time,  effort,  and  resources  are  expended  to  meet  performance  and 
schedule  deadlines  when  compared  to  less  risky  contracts.  To  the  extent  that 
this  is  predicted  early  in  the  program,  it  could  be  useful  to  policy  makers  by 
providing  an  early  warning  that  programs  are  more  likely  to  result  in  cost 
and/or  schedule  overruns. 
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Biggs  (2013)  introduced  the  influence  diagram  in  Figure  1,  which  describes 
the  interactions  between  factors  that  may  be  associated  with  the  occurrence 
of  a  cost  breach.  The  dashed  lines  in  Figure  1  represent  factors  that  must 
be  dealt  with  qualitatively  or  by  using  proxies.  The  solid  lines  represent 
factors  that  can  be  quantitatively  evaluated.  While  the  risk  and  complexity 
of  a  program  may  directly  contribute  to  a  cost  overrun,  the  SE/PM  efforts 
and  the  contract  type  can  influence  the  magnitude  and  frequency  of  cost 
overruns  as  measured  by  cost  breaches. 


Figure  1  indicates  that  the  risk  and  complexity  of  the  MDAP  will  guide  pro¬ 
gram  managers  and  contractors  in  their  selection  of  an  appropriate  contract 
type,  which  in  turn  can  influence  the  government’s  exposure  to  cost  over¬ 
runs.  In  all  likelihood,  the  risk  and  complexity  of  a  program  will  drive  the 
level  of  monitoring  and  negotiation  (transaction  costs)  required  to  manage 
the  program,  and  riskier,  more  complex  programs  will  require  higher  levels 
of  transaction  costs.  Specifically,  we  do  not  expect  that  transaction  costs 
themselves  drive  overruns,  but  rather  that  the  risk  and  complexity  that 
require  higher  levels  of  transaction  costs  drive  breaches. 
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Cost  Breaches 


In  this  article,  we  will  examine  how  the  SE/PM  ratio  and  contract  type 
are  related  to  the  probability  of  incurring  a  cost  breach.  Cost  breaches  occur 
when  the  amount  of  the  cost  overrun  exceeds  certain  parameters  defined 
by  regulation.  Within  the  defense  acquisition  community,  programs  may 
incur  two  types  of  cost  breaches:  Acquisition  Program  Baseline  (APB)  and 
Nunn-McCurdy  breaches.  For  a  program  to  incur  an  APB  breach,  estimated 
program  expenditures  must  be  greater  than  the  APB  EAC  by  at  least  10 
percent.  If  the  difference  is  15  percent  or  more,  a  Nunn-McCurdy  breach  is 
incurred.  Cost  breaches  frequently  are  incurred  in  six  categories  of  appro¬ 
priations:  average  procurement  unit  cost  (APUC);  program  acquisition  unit 
cost  (PAUC);  procurement;  research,  development,  test  and  evaluation 
(RDT&E);  military  construction  (MILCON);  and  acquisition-related  oper¬ 
ations  and  maintenance  (O&M).  Each  of  these  cost  breaches  was  included 
in  the  data  set  for  this  study. 


Within  the  defense  acquisition  community, 
programs  may  incur  two  types  of  cost  breaches: 
Acquisition  Program  Baseline  (APB)  and  Nunn- 
McCurdy  breaches. 

A  formal  cost  estimate  for  a  program  is  normally  developed  at  Milestone 
B,  the  initiation  of  an  acquisition  program,  when  the  APB  is  established. 
The  APB  is  used  for  tracking  and  reporting  cost  and  schedule  for  the  life 
of  the  program,  and  it  includes  threshold  and  objective  values  for  cost, 
schedule,  and  performance.  A  cost  threshold  breach  is  incurred  when  cost 
expenditures  exceed  the  cost  objective  by  10  percent  or  more  (DoD,  2013). 
If  an  MDAP  has  been  officially  rebaselined,1  cost  breaches  are  measured 
to  the  current  baseline. 

Nunn-McCurdy  cost  threshold  breaches  are  based  on  original  cost  esti¬ 
mates  for  PAUC  and  APUC  at  project  completion.  In  the  case  of  a  program 
that  has  rebaselined,  cost  threshold  breaches  are  also  based  on  the  current 
(i.e.,  rebaselined)  cost  estimate  for  PAUC  and  APUC  at  project  completion. 
For  purposes  of  this  study,  a  cost  breach  is  any  reported  in  the  SAR  that  is 
greater  than  or  equal  to  10  percent  above  the  APB.  The  type  of  cost  thresh¬ 
old  breach  and  the  APB  baseline  against  which  it  is  compared  are  shown 
in  Table  1. 
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TABLE  1.  APB  AND  NUNN-MCCURDY  COST  BREACH  THRESHOLDS  j 

APB  Breach 

Nunn-McCurdy 

Nunn-McCurdy 

(RDT&E, 

“Significant” 

“Critical” 

Procurement, 

Breach 

Breach 

MILCON,  O&M) 

(PAUC  &  APUC) 

Current  Baseline 
Estimate 

10% 

+15% 

+25% 

Original  Baseline 
Estimate 

N/A 

+30% 

+50% 

Source:  DoD,  2013 

Figure  2  can  be  used  to  illustrate  cost  overrun  calculations.  The  budgeted 
cost  of  work  performed  (BCWP)  represents  the  total  amount  budgeted  for 
work  packages  that  are  open  or  completed  at  any  given  point  in  time.  The 
budgeted  cost  of  work  scheduled  (BCWS)  represents  the  total  amount 
budgeted  for  the  work  that  was  scheduled  for  completion  at  a  given  point  in 
time.  The  actual  cost  of  work  performed  (ACWP)  is  the  sum  of  actual  costs 
incurred  to  accomplish  the  work  performed  at  a  given  point  in  time. 


FIGURE  2.  EARNED  VALUE  MANAGEMENT 
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The  E  AC  is  the  sum  of  the  AC  WP  and  the  estimate  to 
completion  (ETC)  for  the  remaining  work.  The  ETC 
can  be  calculated  using  the  cost  performance  index 
(CPI)  and  the  schedule  performance  index  (SPI).  The 
formula  for  calculating  ETC  is: 


ETC  =  ( BAC  -  BCWP)  /  ( CPI  *  SPI ) 


(2) 


When  the  EAC,  a  cost  estimate  for  the  total  cost  of 
the  contract,  is  higher  than  the  BAC,  the  baseline 
cost  estimate  of  the  contract,  a  cost  overrun  is  pro¬ 
jected.  Nunn-McCurdy  breaches  are  far  rarer,  perhaps 
because  of  the  political  threat  or  simply  the  higher 
threshold;  thus,  this  article  will  include  analysis  of  the 
potential  of  both  types  of  cost  overruns. 


Data 

This  study  used  data  originally  collected  by  Biggs 
(2013)  from  two  different  data  sources:  SARs  and  the 
Cost  and  Software  Data  Reporting  (CSDR)  System. 
The  SAR  contains  details  of  critical  parameters  of  an 
MDAP,  including  threshold  breaches,  schedule,  per¬ 
formance,  current  contracts,  and  cost  details.  MDAPs 
typically  require  several  contracts  to  be  executed, 
often  concurrently.  SARs  provide  information  for  the 
overall  program  and  not  for  individual  contracts.  A 
SAR  may  list  a  single  contract  or  many  contracts  for  a 
single  MDAP.  Because  threshold  breaches  are  associated  with  contract  esti¬ 
mates,  only  MDAPs  that  listed  one  contract  in  the  “Contracts”  section  of  the 
SAR  were  selected  for  purposes  of  this  study.  In  addition  to  cost  threshold 
breaches,  the  SAR  indicates  the  time  since  program  initiation  at  Milestone 
B,  which  was  used  in  this  study  to  indicate  program  maturity. 


The  program  cost  data  found  in  the  DD  Form  1921  CDSR  provided  by 
the  Defense  Cost  and  Resource  Center  in  the  Defense  Automated  Cost 
Information  Management  System  database  contain  significantly  more 
contract  detail  than  the  SARs.  The  WBS  format  of  the  CDSR  facilitates 
obtaining  information  on  SE/PM  costs.  To  simplify  the  data  collection 
process,  only  the  cost  data  provided  by  the  prime  contractor  were  recorded 
for  further  analysis.  The  SE/PM  cost  values  used  in  this  study  are  extracted 
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from  the  WBS  line  item  values  for  “EAC  SE/PM  cost,”  which  are  listed  on 
the  CDSR  (DD  Form  1921).  The  EAC  SE/PM  cost  is  the  projected  SE/PM 
cost  at  contract  completion.  The  SE/PM  costs  are  inclusive  of  the  total 
contract  costs  less  the  contractor’s  profit/loss  or  fees. 

The  SE/PM  category  reported  by  the  contractor  has  some  limitations.  First, 
the  activities  included  in  this  category  will  vary  somewhat  from  contractor 
to  contractor.  As  a  result,  a  small  portion  of  the  differences  in  SE/PM  costs 
between  contracts  may  be  due  to  differences  between  the  accounting  sys¬ 
tems  used  by  each  contractor.  The  general  category,  however,  is  a  reasonable 
measure  of  the  cost  of  activities  commonly  associated  with  SE/PM.  Second, 
the  costs  included  in  SE/PM  for  a  single  contract  may  vary  over  time  as  new 
costs  are  defined  by  the  contractor  as  being  related  to  SE/PM.  This  could 
explain  a  small  portion  of  the  increase  in  the  SE/PM  cost  in  some  contracts, 
but  we  did  not  observe  significant  differences  in  our  data.  Finally,  we  rec¬ 
ognize  that  the  SE/PM  category  does  not  capture  all  possible  transaction 
costs  nor  was  it  ever  intended  to.  Rather,  it  is  likely  that  many  if  not  most 
of  the  activities  in  this  category  are  related  to  transactions,  as  opposed  to 
production,  and  thus  provide  a  reasonable  measure  of  transaction  costs. 

This  article  will  analyze  the  SE/PM-to-total-cost  ratios  of  MDAPs,  looking 
for  a  potential  correlation  between  these  ratios  and  the  probability  of  expe¬ 
riencing  a  cost  breach.  Determining  the  nature  of  any  potential  relationship 
between  the  SE/PM-to-cost  ratio  and  the  probability  of  cost  breaches  expe¬ 
rienced  by  a  program  will  test  the  hypothesis  that  programs  with  higher  SE/ 
PM  cost  ratios  will  experience  cost  overruns  more  frequently  than  programs 
with  lower  SE/PM  cost  ratios.  More  formally,  we  will  test  the  experimental 
hypothesis  H:  that  the  probability  of  breach  is  increased  by  an  increase  in 
EAC  SE/PM  in  the  total  cost: 

d(Probability  Cost  Breach)  /  d(EAC  SEPM/Total  cost)  >  0 

Hq:  d(Probability  Cost  Breach)  /  d(EAC  SEPM/Total  cost)  =  0 

The  type  of  contract  used  for  the  program  was  also  obtained  from  the 
CDSRs.  Programs  were  noted  as  having  either  firm-fixed-price  type  con¬ 
tracts  or  cost-plus  type  contracts.  The  type  of  contract  used  for  a  program  is 
an  indication  of  the  perceived  level  of  risk  associated  with  execution  of  the 
contract.  As  the  level  of  performance  risk  increases,  the  risk  of  cost  overruns 
also  increases  and  the  amount  of  cost  risk  that  the  contractor  is  willing 
to  assume  tends  to  decrease.  Contract  types  differ  in  how  the  cost  risk  is 
shared  between  the  government  and  the  contractor.  In  a  firm-fixed-price 
contract,  no  cost  sharing  exists  between  the  government  and  the  contractor. 
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and  the  contractor  has  full  responsibility  for  the  performance  costs  and 
resulting  profit  (or  loss).  In  a  cost-plus  contract,  a  share  ratio  based  on  the 
contract  cost  and  the  contractor’s  fee  (profit)  is  negotiated  so  that  the  con¬ 
tractor  has  a  predetermined  responsibility  for  the  performance  costs,  which 
will  directly  affect  the  fee  (profit)  (U.S.  General  Services  Administration, 
2005).  By  including  contract  type  in  our  analysis,  we  can  account  for  basic 
risk  differences  recognized  by  both  the  government  and  the  contractor  at 
the  outset  of  the  program.  Programs  with  aspects  of  both  were  treated  as 
cost-plus  type  contracts  since  cost-plus  contracts  are  abetter  indicator  of 
program  risk. 


1  TABLE  2.  DESCRIPTIVE  STATISTICS  1 

Variable 

Obs 

Mean 

Std.  Dev. 

Min 

Max 

Breach- 
Program  Av. 

32 

0.8125 

1.090649 

0 

4 

Nunn- 

McCurdy 

32 

0.28123 

0.5226715 

0 

2 

Program 

Type 

32 

0.516129 

0.5080005 

0 

1 

EAC  SEPM 
from 

Milestone  B 

32 

13.38844 

11.58719 

0.15 

42.85 

EAC  SEPM 

Program 

Average 

32 

14.66727 

10.68449 

0.87 

43.31 

To  Date 

SEPM 

Program 

Average 

30 

16.59583 

13.81136 

1.4 

54.66667 

Table  2  describes  the  data  used  in  our  study.  The  study  covers  32  programs 
over  84  program  years.  Despite  not  having  greatly  disparate  thresholds,  our 
sample  reflected  far  more  APB  breaches— 26  program  years— versus  only  9 
program  years  for  Nunn-McCurdy  breaches.  Approximately  half  of  the  pro¬ 
grams  were  firm-fixed-price  and  half  were  cost-plus  type  contracts.  We  also 
compared  three  different  measures  of  the  SE/PM  ratio.  The  first  measure  is 
for  the  EAC  of  the  SE/PM  ratio  to  total  costs,  as  estimated  at  Milestone  B. 
SE/PM  are  on  average  about  13  percent  of  total  costs  estimated  at  Milestone 
B.  Programs  that  rebaseline  would  update  this  measure,  but  for  purposes 
of  this  study  we  maintain  the  original  prebreach  measure  because  we  are 
interested  in  the  predictability  of  breaches  based  on  original  assessments  of 
risk.  The  second  measure  is  still  the  EAC,  but  programs  update  this  measure 
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as  they  go  along.  This  number  is  about  1.5  percentage  points  higher  than 
the  Milestone  B  estimate  or  14.7  percent  of  total  costs.  We  also  include  the 
actual  to  date  SE/PM  to  total  cost  at  each  point  in  time  for  our  sample.  At 
16.6  percent  of  total  costs,  this  is  even  higher  than  either  predicted  share, 
indicating  that  estimates  are  on  average  overoptimistic  and  that  SE/PM 
costs  grow,  on  average,  faster  than  total  costs. 

Analysis  of  the  data  shows  that  more  than  half  of  the  MDAPs  in  the  study 
initially  estimated  an  EAC  SE/PM  cost  ratio  of  0.10  or  less  and  experienced 
fewer  than  two  cost  breaches  since  1998.  Furthermore,  it  can  be  inferred 
that  most  of  the  programs  have  experienced  at  least  one  cost  breach,  which 
seems  to  confirm  a  RAND  report  finding  that  most  MDAPs’  actual  costs 
exceeded  baseline  cost  estimates  (Arena,  Leonard,  Murray,  &  Younossi, 
2006).2  Observations  of  the  MDAP  SE/PM-to-total-cost  ratios  agree  with 
the  RAND  study,  suggesting  that  trends  in  SE/PM  costs  vary  across  MDAPs 
(Stem  et  al.,  2006).  Recall  that  SE/PM  costs  are  used  as  proxy  measures  of 
the  transaction  costs  required  to  administer  and  manage  the  MDAP. 


Methodology 

Because  our  data  set  includes  programs  that  have  not  experienced  cost 
breaches  over  the  time  period  studied,  our  data  are  considered  to  be  “right 
censored.”  This  means  that  ordinary  linear  regression  is  not  a  good  option 
for  analyzing  the  data.  Instead,  we  employ  survival  analysis  to  test  whether 
relatively  high  Milestone  B  EAC  SE/PM  is  a  predictor  of  cost  breaches. 
Survival  analysis  is  typically  used  in  medicine  and  social  sciences  to  exam¬ 
ine  when  an  event  of  interest  will  occur.  For  example,  in  medicine  where  the 
event  of  interest  is  a  heart  attack,  we  can  use  survival  analysis  to  predict 
whether  a  patient  will  suffer  a  heart  attack  within  a  period  of  time.  In  this 
study,  the  event  of  interest  is  a  cost  breach,  and  we  are  interested  in  whether 
a  program  will  experience  a  cost  breach. 

In  our  medical  example,  we  could  use  survival  analysis  to  identify  risk  fac¬ 
tors,  such  as  obesity,  that  might  indicate  a  greater  propensity  for  suffering  a 
heart  attack.  In  this  study,  we  are  looking  for  risk  factors  that  might  predict 
cost  breaches.  Two  explanatory  variables  were  included  in  the  analysis: 
EAC  SE/PM  cost  ratio  and  program  contract  type  (fixed-price  or  cost-plus). 
While  the  exact  nature  of  the  relationship  between  cost  threshold  breaches 
and  these  explanatory  variables  is  unknown,  it  is  reasonable  to  presuppose 
that  the  explanatory  variables  influence  the  cost  performance  of  the  MDAPs 
as  shown  in  Figure  1. 
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Using  survival  analysis,  we  construct  a  hazard  function  for  cost  breaches. 
A  hazard  function  shows  (over  time)  the  probability  that  an  event  (such  as 
a  cost  breach)  will  be  incurred.  As  programs  can  experience  multiple 
cost  breaches  despite  rebaselining,  we  allow  for  multiple  breaches 
over  time3  and  estimate  how  the  hazard  of  cost  breach  varies 
with  our  explanatory  variables.  Hazard  models  are  also  useful 
because  they  are  more  tolerant  of  gaps  and  censoring.  Hazard 
models  can  be  thought  of  as  conditional  logits  (Cleves  et  al., 

2010).  We  allow  for  repeat  failures  over  the  period— that 
is,  following  cost  breaches,  we  allow  a  program  to  stay  in 
the  sample. 


Survival  analysis  uses  time-at-risk  as  its  relevant  time 
metric.  Thus,  we  measure  “survival  time”  in  terms  of  the 
maturity  of  the  design  and  technology  of  the  system.  In  this 
study,  program  maturity  is  measured  by  the  time  elapsed 
since  Milestone  B,  the  entry  point  into  the  Engineering  and 
Manufacturing  Development  phase.  For  a  program  to  receive 
approval  to  begin  Milestone  B  in  DoD,  the  design  and  technol¬ 
ogy  associated  with  the  system  must  be  considered  “mature.”4 

In  this  analysis,  we  use  the  Cox-Relative  Hazard.  It  is  con¬ 
sidered  semiparametric  because  it  does  not  imply  a  specific 
functional  form  on  the  hazard  of  breaches  over  time.  The 
proportional  hazard  model  is  specified  as: 


h.  (t)  =  h0  (t)  exp  (xfi) 

which  states  that  the  hazard  a  particular  subject)  faces  at 
time  t  is  a  function  of  the  baseline  hazard  modified  propor¬ 
tionally  by  the  vector  of  regression  coefficients  /?  .  The  Cox 
model  does  not  estimate  the  baseline  hazard.  We  can  convert 
coefficients  from  these  regressions  to  cumulative  hazard  ratios 
to  understand  the  marginal  effect  on  the  baseline  hazard  of  a  change 
in  the  coefficient.  This  is  done  simply  by  calculating  the  exponent  of  the 
coefficient  and  using  it  as  a  multiplier  (e.g.,  the  value  0.9  would  correspond 
with  a  10  percent  reduction,  and  1.1  would  be  a  10  percent  increase).  While 
we  report  coefficients,  we  interpret  our  results  using  exponentiated  hazard 
ratios— that  is,  the  cumulative  hazard  of  a  cost  breach. 
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We  next  examine  whether  baseline  SE/PM-to-cost  ratios  influence  the 
probability  of  a  Nunn-McCurdy  breach  using  a  similar  analysis.  While  the 
difference  between  the  two  is  mainly  the  10  percent  versus  15  percent 
threshold,  Nunn-McCurdy  breaches  are  sufficiently  rare  that  they 
may  have  significantly  different  causes. 


Finally,  we  test  the  robustness  of  our  finding  using  logit  mod¬ 
els.  Logit  is  commonly  used  to  determine  the  influence  of 
exogenous  variables  on  the  probability  of  a  dichotomous 
outcome,  such  as  whether  or  not  a  cost  breach  occurs  in  any 
given  program  year.  Logit  is  preferred  over  a  linear  regres¬ 
sion  model  because,  using  a  logistic  function,  it  constrains 
all  probability-of-occurrence  estimates  to  be  between  0 
and  1.  Formally,  the  logit  model  for  the  probability  can  be 
written  as: 


P(Breach) 


l  +  exf> 


Where  xfi  is  a  function  of  the  vector  of  explanatory  variables. 

We  are  curious  as  to  whether  Milestone  B  SE/PM  predictions 
are  the  best  predictors  of  breaches  or  whether  within-program 
changes  in  SE/PM  estimates  or  actuals  should  cause  concern 
about  program  health.  To  test,  we  use  a  simple  logit  model, 
which  predicts  the  probability  of  failure  on  the  baseline 
EAC  SE/PM  ratios  as  in  our  hazard  model.  We  cluster  the 
standard  errors  by  program.  We  use  a  fixed  effects  (con¬ 
ditional)  logit  to  model  whether  a  change  in  either  EAC  or 
the  actual  SE/PM  cost  ratio  for  an  MDAP  will  change  its 
probability  of  breaching  a  cost  threshold.  Formally,  this 
model  measures  the  impact  of  deviations  by  the  independent 
variable  from  the  program’s  mean  (Allison,  2001). 


Results 

We  find  that  higher  estimated  SE/PM  ratios  are  associated  with  a  higher 
risk  of  APB  breaches.  Figures  3  and  4  show  the  cumulative  risk  of  APB  and 
Nunn-McCurdy  breaches  over  program  maturity.  Table  3  shows  the  results 
for  Hazard  models  for  APB  breaches,  and  Table  4  shows  the  results  for 
Nunn-McCurdy  breaches 
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FIGURE  3.  HAZARD  OF  COST  BREACH 


.t 


FIGURE  4.  HAZARD  OF  NUNN-MCCURDY  BREACH 


.t 
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TABLE  3. 

APB  BREACH  HAZARD 

Variables 

(1) 

(2) 

EAC  SE/PM  from 

0.0482*** 

0.0284* 

Milestone  B 

(0.0147) 

(0.0168) 

Type 

1.125** 

(0.552) 

Observations 

84 

84 

Note.  Standard  errors  in  parentheses 

***  p<0.01,  **  p<0.05,  *  p<0.1 

In  the  first  model  a  1  percentage  point  increase  in  the  estimated  SE/PM 
ratio  at  completion  increases  the  risk  of  breach  by  5  percent,  a  result  that  is 
statistically  significant  at  the  1  percent  level.  When  contract  type  is  added 
as  a  control,  the  impact  of  the  SE/PM  ratio  goes  down  to  3  percent  and  its 
significance  is  reduced  to  the  10  percent  level.  Looking  at  the  impact  of 
contract  type,  we  find  that  having  a  cost-plus  type  contract  multiplies  the 
risk  of  an  APB  breach  by  3.1,  which  is  significant  at  the  5  percent  level. 


|  TABLE  4.  NUNN- 

MCCURDY  BREACH  HAZARD 

Variables 

(3) 

(4) 

EAC  SE/PM  from 

0.0247 

0.00352 

Milestone  B 

(0.0268) 

(0.0290) 

Type 

1.269 

(0.886) 

Observations 

84 

84 

Note.  Standard  errors  in  parentheses 

***  p<0.01,  **  p<0.05,  *  p<0.1 

Nunn-McCurdy  breaches  are  significantly  less  common  than  APB  breaches 
and  SE/PM  ratios  or  contract  type  do  not  appear  to  determine  them.  The 
SE/PM  ratio  is  not  a  significant  predictor  of  Nunn-McCurdy  breaches,  and 
while  type  has  a  similar  magnitude  to  its  impact  on  APB  breaches,  it  is  not 
statistically  significant  either. 
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1  TABLE  5.  LOGIT  ANALYSIS  | 

Variables 

(1) 

Logit 

(2) 

Fixed  Effects 

(3) 

Fixed  Effects 

Baseline  EAC  SE/PM 

0.0433* 

(0.0244) 

To  Date  SE/PM 

-0.0358 

(0.0491) 

EAC  SE/PM 

-0.0924 

(0.0782) 

Constant 

-1.433*** 

(0.508) 

Observations 

84 

36 

39 

Number  of  programs 

10 

11 

Table  5  shows  the  results  of  logit  analysis  for  APB  breaches.  We  find  that 
the  likelihood  of  a  cost  breach  is  related  to  the  EAC  SE/PM  ratio  predicted 
at  milestone  B,  but  not  to  changes  in  the  predicted  ratio  or  to  the  actual  SE/ 
PM  ratio.  The  marginal  effect  of  the  logit  model  is  very  similar  to  the  hazard 
function  and  statistically  significant  at  the  10  percent  level:  for  every  1  per¬ 
cent  increase  or  decrease  in  the  EAC  SE/PM  cost  ratio,  there  is  an  increase 
or  decrease  of  4  percent  in  the  probability  of  a  program  sustaining  a  cost 
threshold  breach.  We  do  not  find  any  within-program  impact  using  fixed 
effects  logit  models,  indicating  that  a  change  in  the  EAC  SE/PM  ratio  is  not 
an  indicator  of  a  potential  breach.  Thus  forecasts  of  cost  breaches  should 
focus  on  initial  SE/PM  ratio  estimates. 


Conclusion 

This  article  successfully  tested  the  hypothesis  that  transaction  costs  could 
help  explain  future  cost  breaches  of  MDAPs.  Using  SE/PM  costs  as  a  ratio  of 
total  program  costs,  we  find  the  greater  this  ratio  is  at  the  outset  (Milestone 
B  estimate),  the  greater  the  risk  of  eventual  cost  breaches.  This  information 
reflects  the  program  manager’s  implicit  assessment  of  the  risk  of  a  program 
and  can  be  a  valuable  early  indicator  of  which  programs  will  benefit  from 
greater  oversight.  We  should  be  careful  to  note  that  high  SE/PM  ratios  may 
be  the  result  of  program  managers  responding  proactively  to  program  risks. 
As  such,  we  are  not  suggesting  that  high  SE/PM  ratios  are  abad  thing— simply 
that  they  can  be  used  to  predict  cost  breaches,  which  often  occur  in  high-risk, 
more  complex  programs.  This  suggests  that  the  SE/PM  ratio  is  a  measurable 
indicator  of  cost  risk  and  anticipated  transaction  costs. 


72 


Defense ARJ,  January  2016,  Vol.  23 No.  1 : 56-76 


January  2016 


This  article  also  offers  an  innovative  way  of  looking  at  cost  breaches,  using  sur¬ 
vival  analysis  to  forecast  cost  breaches  in  MDAPs  over  a  program’s  life  cycle. 
We  offer  somewhat  robust  analysis,  using  a  more  traditional  logit  analysis,  of 
our  predictions.  We  believe  this  is  a  step  forward  in  using  measures  available 
in  DoD’s  vast  databases  of  cost  information  to  develop  more  robust  forecasts 
of  potential  cost  overrun  risks  in  programs.  These  findings  suggest  that  the 
department  could  benefit  from  capturing  more  explicit  measures  of  transac¬ 
tion  costs  to  determine  more  precisely  their  role  in  predicting  cost  variability. 


Appendix 

PROGRAMS  SELECTED  FOR  STUDY 

Active  Electronically  Scanned  Array  (AESA)  Radar 
AIM-9X/Short  Range  Air-to-Air  Missile 

AIM-120  Advanced  Medium  Range  Air-to-Air  Missile  (AMRAAM) 

Airborne  and  Maritime/Fixed  Station  Joint  Tactical  Radio  System  (AMF  JTRS) 
AN/WQR-3,  Advanced  Deployable  System  (ADS) 

Apache  Block  IIIA  Remanufacture  (AB3A  REMANUFACTURE) 

AV-8B/Attack,  V/STOL,  Close  Air  Support  (Harrier  11+  Remanufacture) 

B-2  Radar  Modernization  Program 

Cobra  Judy  Replacement  (Cobra  Judy  Replacement) 

EA-18G  Growler  (EA-18G) 

Expeditionary  Fighting  Vehicle  (EFV) 

E-3  AWACS  Radar  System  Improvement  Program  (RSIP) 

E-2C  Reproduction 

Family  of  Advanced  Beyond  Line-of-Sight  Terminals  (FAB-T) 

Family  of  Medium  Tactical  Vehicles  (FMTV) 

Guided  Multiple  Launch  Rocket  System/DPICM/Unitary/Alternative  Warhead 
(GMLRS/GMLRS  AW) 

Joint  Common  Missile  (JCM) 

Joint  Tactical  Radio  System  Ground  Mobile  Radio  (formerly  Cluster  1)  (JTRS  GMR) 
Longbow  Hellfire  -  subsystem  of  the  AH-64  Apache  Weapon  System 
LHA  Replacement  Amphibious  Assault  Ship 

MQ-4C  Unmanned  Aircraft  System  Broad  Area  Maritime  Surveillance  (MQ-4C 
UAS  BAMS) 

Multi-Platform  Radar  Technology  Insertion  Program  (MP-RTIP) 

National  Polar-orbiting  Operational  Environmental  Satellite  System  (NPOESS) 
Presidential  Helicopter  Replacement  (VH-71)  Program 
P-8A  Poseidon 

Sense  and  Destroy  Armor  (SADARM) 

Small  Diameter  Bomb  Increment  II  (SDB  II) 

Space  Based  Infrared  System  (SBIRS)  High  Program 
Standard  Missile  (SM)  -  2  Block  IV 
Stryker  Family  of  Vehicles  (STRYKER) 

UH-72A  Light  Utility  Helicopter  (LUH) 

Warfighter  Information  Network  -  Tactical  (WIN-T) 
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Endnotes 

'Circumstances  authorizing  changes  to  the  baseline  are  limited;  revisions  to  the 
current  APB  are  not  authorized  unless  there  is  a  significant  change  in  program 
parameters  and  must  be  approved  by  the  Milestone  Decision  Authority  (DoD,  2013). 

2  For  most  of  the  programs  reviewed,  actual  costs  exceeded  the  baseline  cost 
estimate  established  at  Milestone  B  (program  initiation),  as  measured  by  the  cost 
growth  factor  (Arena  et  al.,  2006). 

3  For  robustness,  we  also  ran  the  analysis  without  allowing  for  multiple  breaches,  and 
while  the  results  were  weaker,  they  were  still  statistically  significant  and  of  a  similar 
magnitude.  Still,  the  fact  that  including  multiple  breaches  strengthens  our  results, 
indicates  that  programs  with  high  levels  of  complexity  and  risk  often  suffer  for  these 
high  levels  repeatedly. 

4Milestone  B  approval  authorizes  an  MDAP  to  enter  the  Engineering  and 
Manufacturing  Development  phase  of  the  acquisition  process.  Statutory  requirements 
for  MDAPs  to  achieve  Milestone  B  approval  are  found  in  Title  10  U.S.C.  §  2366b. 

These  requirements  stipulate  that  the  program  be  certified  by  the  Milestone  Decision 
Authority  to  be  affordable,  fully  funded  through  the  Future  Years  Defense  Program 
(FYDP),  and  that  the  cost  and  schedule  estimates  are  reasonable. 
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This  investigative  study  demonstrates  the  benefits  of  addressing  human 
considerations  early  in  the  system  development  life  cycle  that  will  bring  long¬ 
term  benefit  to  program  managers  and  systems  engineers.  The  approach  used 
a  retrospective  content  analysis  of  documents  from  weapon  systems  acquisi¬ 
tion  programs,  namely  Major  Defense  Acquisition  Programs.  Binary  logistic 
regression  analyses  were  conducted  to  predict  the  effect  of  the  presence  of 
words  relating  to  Human  Systems  Integration  on  the  success  of  programs. 
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This  investigative  study  corroborates  the  idea  that  some  benefit  may 
be  derived  from  implementing  Human  Systems  Integration  during  the 
weapon  systems  acquisition  life  cycle. 
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In  1981,  U.S.  congressional  watchdogs  recommended  improving  weapon 
systems  design  by  addressing  human  consideration  problems  early  during 
system  acquisition  (U.S.  General  Accounting  Office,  1981).  Today,  the  pro¬ 
cess  by  which  human  considerations  are  included  in  the  planning  and 
implementation  of  a  system  is  known  as  Human  Systems  Integration  (HSI) 
(International  Council  on  Systems  Engineering  [INCOSE],  2012).  The  ben¬ 
efit  of  including  HSI  in  weapon  systems  design  and  acquisition  is  best 
realized  by  giving  HSI  early  attention  and  priority  during  the  planning  stage. 
This  article  will  demonstrate  that  a  decreased  percentage  of  HSI  words  in 
documents  originated  during  acquisition  will  coincide  with  unforeseen 
costs,  delays,  and  performance  problems. 


The  benefit  of  including  HSI  in  weapon  systems  design 
and  acquisition  is  best  realized  by  giving  HSI  early 
attention  and  priority  during  the  planning  stage. 


In  fact,  HSI-related  content  in  acquisition  documents  may  influence  pro¬ 
gram  success.  Typically,  a  program  is  considered  successful  if  it  avoids 
cost  overruns,  avoids  performance  breaches,  or  avoids  schedule  breaches. 
Systems  Engineering  (SE)  is  the  interdisciplinary  approach  for  develop¬ 
ing  systems  (INCOSE,  2012).  HSI  is  an  important  part  of  SE,  and  thus  the 
acquisition  life  cycle  (Karwowski,  2012).  Understandably,  the  decisions  that 
program  managers  and  systems  engineers  make  early  in  the  acquisition  life 
cycle  will  affect  program  success  and  life-cycle  costs.  For  example,  to  help 
organizations  with  incorporating  HSI  into  their  design  process,  Handley 
and  Knapp  (2014)  have  created  the  Human  Viewpoint  tool  for  early  imple¬ 
mentation  of  HSI  into  the  acquisition  life  cycle.  Ahram  and  Karwowski 
(2012)  warn  that  failing  to  address  costs  related  to  human  performance  (e.g., 
Human  Total  Ownership  Cost)  early  in  the  life  cycle  will  lead  to  schedule 
overruns,  diminished  system  performance,  inadequate  training,  and  mis¬ 
aligned  plans  for  manpower  and  personnel  allocation.  Cramer,  Sudhoff,  and 
Zivi  (2011)  posit  that  integrating  survivability  as  a  design  objective  early  in 
the  life  cycle  can  benefit  the  design  process.  Assessing  human  capabilities 
during  technology  readiness  level  evaluation,  according  to  Wallace,  Bost, 
Thurber,  and  Hamburger  (2007),  can  help  the  program  avoid  cost  overruns. 
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The  interaction  between  humans  and  the  systems  they  use  affects  program 
success,  as  well  as  life-cycle  costs.  The  documents  from  weapon  systems 
acquisition  programs,  namely  major  defense  acquisition  programs  (MDAP), 
contain  a  history  of  each  program’s  system  development  life  cycle,  and  this 
history  indicates  what  considerations  were  involved  in  the  system  devel¬ 
opment  life  cycle.  It  follows  then  that  HSI-related  content  in  acquisition 
documents  is  interrelated  with  program  success. 

This  article  is  essentially  an  investigative  study  or  retrospective  content 
analysis  of  MDAP  documents.  The  author’s  goal  is  to  present  a  sound  argu¬ 
ment  that  omitting  HSI  during  weapon  systems  acquisition  will  coincide 
with  acquisition  life-cycle  cost  overruns,  as  well  as  schedule  slippages  and 
performance  breaches.  More  specifically,  this  investigative  study  addresses 
the  gap  in  knowledge  among  weapon  systems  acquisition,  HSI,  and  acquisi¬ 
tion  life-cycle  cost,  performance,  and  schedule  (Figure  1). 


FIGURE  1.  VENN  DIAGRAM  OF  INTERRELATIONSHIPS  AMONG 
WEAPON  SYSTEMS,  HSI,  AND  LIFE-CYCLE  COST, 
PERFORMANCE,  AND  SCHEDULE 


Performance,  and 
Schedule 
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Problem  Statement 

Stakeholders,  such  as  program  managers  and  systems  engineers,  strive 
to  mitigate  unforeseen  costs  during  the  system  development  life  cycle.  One 
way  to  achieve  this  objective  is  to  prioritize  human  considerations  early  in 
the  system  development  life  cycle.  A  program  manager  or  systems  engineer 
could  predict  program  success  using  regression  analysis  of  historical  data. 
Program  documents,  such  as  Selected  Acquisition  Reports  (SARs),  provide  a 
valuable  source  of  historical  data  about  weapon  systems  programs  (Assidmi, 
Sarkani,  &  Mazzuchi,  2012;  Bielecki  &  White,  2005;  Birchler,  Christie,  & 
Groo,  2011).  For  purposes  of  this  investigative  study,  logistic  regression  and 
HSI-related  terminology  in  documents  will  be  used  to  make  predictions 
about  program  success. 

Presence  of  HSI-related  terminology  is  defined  by  the  percentage  of  HSI- 
related  words  per  document.  Program  success  is  defined  by  avoidance  of  cost 
overrun,  avoidance  of  performance  breach,  or  avoidance  of  schedule  breach. 
Overall,  the  objective  is  to  conduct  a  retrospective  content  analysis  of  MDAP 
documents  as  an  approach  by  which  to  seek  the  presence  of  HSI-related 
terminology  in  weapon  systems  acquisition.  This  approach  is  designed  to 
demonstrate  that  an  earlier  presence  of  HSI-related  terminology  will  pre¬ 
dict  better  outcomes  for  weapon  systems  acquisition  programs  with  money 
saved,  time  saved,  and  good  performance.  A  definitive  research  question 
that  addresses  the  problem  identified  by  this  investigative  study  follows. 

Does  the  percentage  ofHSI  words  within  the  document  coin¬ 
cide  with  unforeseen  cost  overruns,  performance  breaches, 
and  schedule  slippages? 

Although  the  findings  of  this  investigative  study  did  not  yield  strongly 
predictive  regression  models,  significant  findings  emerged  that  suggest 
schedule  slippages  and  cost  overruns  may  be  associated  with  a  reduction 
of  HSI-related  terminology.  From  the  findings,  the  presence  of  terminology 
about  human  factors  engineering,  habitability,  and  survivability  as  well 
as  manpower,  personnel,  and  training  suggests  that  a  program  will  likely 
succeed.  This  finding  corroborates  the  idea  that  a  solution  to  the  problem— 
specifically  saving  time,  saving  money,  and  improving  performance— will 
be  the  inclusion  of  HSI-related  content  early  in  the  weapon  systems  acqui¬ 
sition  life  cycle. 
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Data  Collection 

The  data  collection  effort  identified  HSI-related  terminology  in  each 
document,  including  HSI  word  percentages.  Program  success  data  were 
collected  with  regard  to  cost,  schedule,  and  performance.  Weapon  systems 
acquisition  programs,  specifically  MDAPs,  were  also  identified.  Additional 
data  were  collected  to  identify  when  each  document  was  published  with 
regard  to  its  corresponding  program’s  Milestone  B. 

HSI-Related  Terminology 

As  shown  in  Table  1,  the  HSI-related  terminology  in  this  investigative 
study  consisted  of  words  that  refer  to  the  nine  HSI  domains  defined  by 
the  Department  of  the  Air  Force  (2014).  Also  included  were  the  terms  HSI 
and  MANPRINT,  which  are  synonymous  (Drillings,  2014).  It  is  helpful 
to  note  that  HSI  is  defined  differently  among  organizations.  For  example. 
Headquarters  Department  of  the  Army  (2014)  defined  seven  HSI  domains: 


1.  manpower 

2.  personnel  capabilities 

3.  training 

4.  human  factors  engineering 

5.  system  safety 

6.  health  hazards 

7.  soldier  survivability 


Department  of  the  Navy  (2009)  defined  seven  slightly 
different  HSI  domains: 


Department  of  the  Air  Force  (2014),  however,  defined  nine 
versus  seven  HSI  domains: 


1. 

2. 

3. 

4. 

5. 

6. 
7. 


manpower 
personnel 
training 

human  factors  engineering 
environmental  safety  and  occupational  health 
habitability 
personnel  survivability 


1. 

2. 

3. 

4. 


manpower 

personnel 

training 

environment 
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5.  safety 

6.  occupational  health 

7.  human  factors  engineering 

8.  survivability 

9.  habitability 


This  investigative  study  refers  to  the  most  complete  list  of  HSI  domains,  as 
identified  by  Department  of  the  Air  Force  (Table  1). 


TABLE  1.  HSI-RELATED  WORDS  AND  CORRESPONDING  HSI  DOMAINS 

HSI  Domain 

HSI-Related  Words 

NA 

HSI,  Manpower  and  Personnel  Integration 
(MANPRINT) 

Human  Factors  Engineering 

fatigue,  human,  people,  perform, 
performance,  performed,  performing, 
performs,  situational  awareness,  troops, 
usability,  utility,  workload 

Habitability 

habitability,  shelter 

Survivability 

survivability 

Environment 

environment,  environmental 

Safety 

protect,  protected,  protection,  protective, 
safety,  secure,  security 

Occupational  Health 

health 

Manpower 

interoperability,  maintain,  maintainability, 
maintained,  maintained  maintaining, 
maintains,  maintenance,  manpower,  manned, 
manning,  operability,  operate,  operated, 
operates,  operating,  operational 

Personnel 

infantry,  manage,  managed,  management, 
manager,  manages,  managing,  personnel, 
pilot,  role,  staff,  warfighter,  warrior 

Training 

instructor,  train,  trained,  training 

To  indicate  HSI-related  terminology  within  each  program  document,  data 
consisted  of  word  percentages  for  each  HSI-related  word  of  interest  for  each 
of 546  program  documents.  These  word  percentages  were  calculated  using 
word  counts  for  each  HSI-related  word  of  interest  and  the  total  word  count 
for  each  document.  Table  2  shows  the  number  of  documents  per  program 
and  the  range  of  word  counts  per  program. 
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1  TABLE  2.  MAJOR  DEFENSE  ACQUISITION  PROGRAMS  [ 

Program  Name 

Total 

Documents 

Range  of 
Word  Counts 

Advanced  Deployable  System 

7 

504  to  1,237 

DDG  1000  Zumwalt  Class  Destroyer 

37 

107  to  17,317 

F-22  Raptor  Advanced  Tactical  Fighter 
Aircraft 

71 

86  to  9,378 

F-35  Joint  Strike  Fighter  Aircraft 

58 

68  to  17,700 

Ground/Air  Task-Oriented  Radar 

6 

228  to  3,807 

EA-18G  Growler  Aircraft 

35 

89  to  39,656 

F/A-18E/F  Super  Hornet 

12 

801  to  2,574 

Joint  Land  Attack  Cruise  Missile  Defense 
(JLACMD)  Elevated  Netted  Sensor  System 

16 

161  to  31,314 

Joint  Tactical  Radio  System 

45 

275  to  5,197 

Littoral  Combat  Ship 

27 

73  to  64,661 

Mine  Resistant  Ambush  Protected  Vehicle 

48 

115  to  51,790 

P-8A  Poseidon  Multi-Mission  Maritime 
Aircraft 

22 

178  to  31,658 

Space  Based  Infrared  System  High 

48 

194  to  4,390 

V-22  Osprey  Joint  Services  Advanced 
Vertical  Lift  Aircraft 

47 

117  to  9,713 

Vertical  Take-off  and  Landing  Tactical 
Unmanned  Aerial  Vehicle 

16 

150  to  2,721 

Warfighter  Information  Network-Tactical 

51 

228  to  31,132 

Data  were  collected  from  the  546  documents  and  entered  into  SPSS 
Statistics  Version  22.0  for  Windows.  HSI  words  within  the  sampled  doc¬ 
uments  ranged  from  zero  to  2,262.  The  average  number  of  HSI  words  was 
42.60  ( SD  =  160.73).  Total  words  for  the  sampled  documents  ranged  from  68 
to  64,661,  and  the  total  word  count  was  2,010.53  on  average  (SD  =  5,109.40). 

HSI-related  words  were  separated  into  three  categories  (Table  3).  Some 
overlap  occurred  among  each  of  the  HSI  domains,  and  some  words  fit  into 
more  than  one  domain  description.  Typically,  environmental,  safety,  and 
occupational  health  issues  are  grouped  together  and  identified  with  the 
acronym  ESOH,  as  are  manpower,  personnel,  and  training  issues,  which 
are  identified  as  MPT.  For  this  investigative  study,  these  same  groups 
were  identified,  thus  the  data  included  ESOH  and  MPT  word  percentages. 
Because  the  terms  habitability  and  survivability  generated  a  small  quantity 
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of  words,  they  were  grouped  together  with  Human  Factors  Engineering 
(HFE),  along  with  the  terms  HSI  and  MANPRINT,  thus  the  data  included 
HFE/Hab/Surv  word  percentages. 


TABLE  3.  HSI-RELATED  WORDS  AND  CORRESPONDING 

HSI  CATEGORIES 

HSI  Category 

HSI-Related  Words 

HFE/Hab/Surv 

HSI,  MANPRINT,  fatigue,  habitability,  human,  people, 
perform,  performance,  performed,  performing, 
performs,  shelter,  situational  awareness,  survivability, 
troops,  usability,  utility,  workload 

ESOH 

environment,  environmental,  health,  protect,  protected, 
protection,  protective,  safety,  secure,  security 

MPT 

infantry,  instructor,  interoperability,  maintain, 
maintainability,  maintained,  maintained  maintaining, 
maintains,  maintenance,  manage,  managed, 
management,  manager,  manages,  managing,  manpower, 
manned,  manning,  operability,  operate,  operated, 
operates,  operating,  operational,  personnel,  pilot,  role, 
staff,  train,  trained,  training,  warfighter,  warrior 

Success  Metrics 

Program  success  data  were  collected  to  investigate  each  MDAP’s  cost 
overruns,  schedule  slippages,  and  performance  breaches.  These  success 
metrics  were  (a)  SAR- identified  cost  breaches,  (b)  SAR- identified  schedule 
breaches,  (c)  SAR-identified  performance  breaches,  (d)  GAO  assessment 
indicating  total  program  over  budget,  (e)  GAO  assessment  indicating  pro¬ 
gram  unit  cost  increase,  and  (f)  Weapon  Book  program  amount  spent  went 
over  budget.  None  of  these  six  metrics  had  absolutely  complete  data  because 
data  were  not  available  for  each  program  for  each  fiscal  year.  To  collect 
ample  data,  all  six  metrics  were  considered  for  this  investigative  study. 

MDAPs 

Several  factors  were  considered  in  the  selection  of  the  sample  of  MDAPs 
analyzed  in  this  study.  Each  program  needed  sufficient  documentation  for 
the  HSI  word  analysis  as  well  as  cost,  schedule,  and  performance  data.  Table 
4  lists  the  16  MDAPs  alongside  their  common  names.  MDAP  documents 
were  collected  between  June  2013  and  October  2014,  primarily  from  the 
Defense  Acquisition  Management  Information  Retrieval  (DAMIR)  data¬ 
base  (DAMIR,  n.d.).  Additional  MDAP  documents  were  collected  from  the 
Acquisition  Decision  Memoranda  (ADM)  Web  site  (ACQWeb)  (Acquisition 
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Decision  Memoranda,  2014).  The  objective  was  to  acquire  acquisition  pro¬ 
gram  documents  that  are  consistent  from  program  to  program;  the  DAMIR 
database  and  ADM  Web  site  made  this  possible. 


TABLE  4.  HSI-RELATED  WORDS  AND  CORRESPONDING  HSI  DOMAINS  1 

Program  Name 

Common  Name 

Advanced  Deployable  System 

ADS 

DDG  1000  Zumwalt  Class  Destroyer 

DDG  1000 

F-22  Raptor  Advanced  Tactical  Fighter  Aircraft 

F-22 

F-35  Joint  Strike  Fighter  Aircraft 

JSF 

Ground/Air  Task  Oriented  Radar 

G/ATOR 

EA-18G  Growler  Aircraft 

EA-18G 

F/A-18E/F  Super  Hornet 

F/A-18E/F 

Joint  Land  Attack  Cruise  Missile  Defense  (JLACMD) 
Elevated  Netted  Sensor  System 

JLENS 

Joint  Tactical  Radio  System 

JTRS 

Littoral  Combat  Ship 

LCS 

Mine  Resistant  Ambush  Protected  Vehicle 

MRAP 

P-8A  Poseidon  Multi-Mission  Maritime  Aircraft 

P-8A 

Space  Based  Infrared  System  High 

SBIRS  High 

V- 22  Osprey  Joint  Services  Advanced  Vertical  Lift 
Aircraft 

V-22 

Vertical  Take-off  and  Landing  Tactical  Unmanned  Aerial 
Vehicle 

VTUAV  Fire 

Scout 

Warfighter  Information  Network-Tactical 

WIN-T 

Note.  If  a  program  had  any  increments,  then  the  increment  was  noted,  and  appropriate 
data  for  that  increment  were  collected. 

None  of  the  MDAP  documents  identifying  HSI-related  terminology  were 
used  to  obtain  cost,  schedule,  or  performance  data.  In  addition  to  collecting 
MDAP  documents,  other  sources  of  data  were  collected  to  identify  program 
cost  overruns,  schedule  slippages,  and  performance  breaches  (Table  5). 
Furthermore,  none  of  the  documents  that  were  used  for  obtaining  infor¬ 
mation  about  program  cost  overruns,  schedule  slippages,  and  performance 
breaches  were  used  to  obtain  HSI  data. 
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j  TABLE  5.  BREACH  DATA  SOURCES  1 

Breach  Data  Source  and  Description 

Types  of 
Breaches 

Origin 

Selected  Acquisition  Reports  (SAR) 
indicate  whether  or  not  a  program  has 
experienced  cost  breach,  schedule 
breach,  or  performance  breach.  Data 
indicating  breaches  were  assigned  Is 
for  breach  occurrence  and  Os  for  no 
breach. 

cost  overruns, 
schedule 
slippages,  and 
performance 
breaches 

DAMIR  database 

(DAMIR,  n.d.) 

Government  Accountability  Office 
(GAO)  assessments  for  major 
weapon  programs  report  whether  a 
program  has  gone  over  budget  and 
whether  a  program’s  unit  cost  has 
increased.  Data  indicating  that  the 
total  program  had  gone  over  budget 
were  assigned  Is,  as  well  as  data 
indicating  that  the  program’s  unit 
cost  had  increased;  absences  of  these 
conditions  were  assigned  Os. 

cost  overruns 

GAO  Web  site 

(GAO,  2014) 

The  Comptroller’s  Weapon  Books 
report  the  Budget  Request  amounts 
that  a  program  requested  for  a 
given  fiscal  year  and  later  what  the 
program  actually  spent  during  that 
fiscal  year.  A  cost  overrun  can  be 
determined  from  comparing  what 
amount  was  requested  and  then  what 
amount  was  actually  spent.  Is  were 
assigned  to  indicate  that  a  program 
went  over  budget  for  the  given  fiscal 
year,  and  Os  were  assigned  in  the 
absence  of  this  condition. 

cost  overruns 

Comptroller’s 

Web  site 

(Under  Secretary 
of  Defense 
Comptroller,  2014) 

Experimental  Design 

Because  the  regression  analysis  was  intended  to  measure  six  dependent 
variables,  six  analyses  were  conducted  for  each  of  the  six  dependent  vari¬ 
ables:  (a)  SAR  cost  breach,  (b)  SAR  performance  breach,  (c)  SAR  schedule 
breach,  (d)  GAO  total  program  over  budget,  (e)  GAO  program  unit  cost 
increase,  and  (f)  Weapon  Book  amount  spent  over  budget.  Subcategories 
of  HSI  word  percentage  per  MDAP  document  included  (a)  ESOH  word  per¬ 
centage,  (b)  HFE/Hab/Surv  word  percentage,  and  (c)  MPT  word  percentage. 
The  independent  variables  were  the  subcategories  of  HSI  word  percentage 
per  MDAP  document. 
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Data  Analysis 

As  shown  in  Table  5,  qualitative  breach  data  were  categorized  with 
Os  and  Is.  When  dependent  variables  are  qualitative,  a  logistic  regression 
equation  can  be  used  to  create  a  model  of  the  probability  that  the  dependent 
variable’s  value  will  be  either  0  or  1  (Chatterjee  &  Hadi,  2012).  After  the 
regression  model  has  been  created,  the  data  are  compared  to  the  model  to 
discover  what  ratio  of  the  data  was  classified  0  or  1  correctly  (Chatterjee  & 
Hadi,  2012).  Binary  logistic  regression,  the  data  analysis  method  selected 
for  this  data  set’s  interpretation,  is  a  method  for  modeling  probabilities  when 
the  outcome  falls  between  0  and  1. 


Because  each  budget  for  each  MDAP  is  unique  from  the  budgets  of  other 
MDAPs,  the  numeric  values  for  each  budget  would  consist  of  different 
numbers  that  cannot  be  directly  compared.  To  assess  the  data,  one  may  ask 
the  question,  “Did  an  MDAP  go  over  budget,  or  didn’t 
it?”  Answering  this  question  opens  the  possibility 
to  make  the  data  dichotomous  with  Os  for  no 
and  Is  for  yes.  These  dichotomous  data  are 
qualitative  because  they  yield  a  qualitative 
value,  such  as  good  or  bad  (Chatterjee  & 

Hadi,  2012).  In  this  manner,  data  regard¬ 
ing  whether  or  not  there  was  a  breach 
can  be  assigned  (categorized)  Os  and  Is, 
and  the  appropriate  statistical  method  is 
binary  logistic  regression. 

Shown  here  is  the  model  for  the  predic¬ 
tive  formulae  that  were  the  outcome  of  this 
study’s  analysis.  Because  predictive  formulae 
from  logistic  regression  are  nonlinear,  they  must  be 
transformed.  The  probability  of  something  happening,  such  as  a  cost  over¬ 
run,  is  the  odds  ratio.  The  logit  is  identified  by  finding  the  logarithm  of  the 
odds  ratio.  This  transformation  ensures  that  the  predictive  formulae  will 
be  linear  (Chatterjee  &  Hadi,  2012). 


logit[Pr(Y=Unforeseen  cost  overrun,  performance 
breach,  or  schedule  slippage)]  =  BQ  +/-  B^ESOH  word 
percentage)  +/-  B2(HFE/Hab/Surv  word  percentage) 
+/-  B3(MPT  word  percentage) 
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Results 

To  examine  the  hypothesis,  a  series  of  binary  logistic  regressions  was 
conducted.  Due  to  the  exploratory  nature  of  this  investigative  study,  an 
alpha  level  of  0.10  was  employed  for  assessment  of  statistical  significance. 
The  hypothesis  is  shown  here. 

The  percentage  ofHSI  words  within  the  document  will  coin¬ 
cide  with  unforeseen  cost  overruns,  performance  breaches, 
and  schedule  slippages. 

Results  of  the  six  regression  analyses  are  presented  in  Table  6,  while  regres¬ 
sion  coefficients  are  presented  in  Table  7.  Four  of  the  six  regressions  yielded 
significant  models.  SAR  schedule  breach,  GAO  total  program  over  budget, 
GAO  program  unit  cost  increase,  and  Weapon  Book  amount  spent  over 
budget  were  associated  with  either  ESOH  word  percentage,  HFE/Hab/Surv 
word  percentage,  or  MPT  word  percentage.  These  four  regressions  were 
further  analyzed  to  assess  the  hypothesis. 
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TABLE  6.  MODEL  FIT  FOR  SIX  BINARY  LOGISTIC 

REGRESSION  ANALYSES 

Dependent  Variable 

X2(  7) 

P 

Nagelkerke  R 

1. 

SAR  cost  breach 

9.15 

.242 

.03 

2. 

SAR  performance  breach 

5.90 

.552 

.07 

3. 

SAR  schedule  breach* 

20.21 

.005 

.06 

4. 

GAO  total  program  over  budget* 

28.00 

<  .001 

.13 

5. 

GAO  program  unit  cost  increase* 

22.29 

.002 

.12 

6. 

Weapon  Book  amount  spent  over 
budget  * 

23.83 

.001 

.14 

Note.  An  asterisk  (*)  indicates  significance  at  the  p  < 

.10  level. 

TABLE  7.  MODEL  COEFFICIENT  DETAILS  FOR 
BINARY  LOGISTIC  REGRESSIONS 


Regression  B  SE  Wald  p  O.R. 

SAR  schedule  breach  + 

MPT  word  -0.74  0.34  4.75  .029  0.48 

percentage 

GAO  total  program  over  budget + 

MPT  word  -0.98  0.59  2.78  .095  0.38 

percentage 

GAO  program  unit  cost  increase  f 


MPT  word 
percentage 

-1.05 

0.51 

4.23 

.040 

0.35 

Weapon  Book  amount  spent  over  budget  + 

ESOH  word 
percentage 

5.23 

2.86 

3.35 

.067 

187.23 

HFE/Hab/Surv  word 
percentage 

-4.90 

1.77 

7.64 

.006 

0.01 

MPT  word 
percentage 

1.06 

0.64 

2.74 

.098 

2.87 

Note.  A  dagger  (+)  indicates  model  significance  at  the  p  < 

.10  level. 

To  examine  the  hypothesis,  the  three  predictors  (ESOH  word  percentage, 
HFE/Hab/Surv  word  percentage,  and  MPT  word  percentage)  of  the  four  sig¬ 
nificant  models  were  examined  (Table  6).  Model  coefficients  are  presented 
in  Table  7.  For  SAR  schedule  breach,  MPT  word  percentage  was  a  significant 
predictor  ( B  =  -0.74,  p  =  .029,  OR  =  0.48),  suggesting  that  as  the  percentage  of 
MPT  words  increased,  the  likelihood  of  a  SAR  schedule  breach  decreased. 
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For  GAO  total  program  over  budget,  MPT  word  percentage  was  a  significant 
predictor  ( B  =  -0.98,  p  =  .095,  OR  =  0.38),  suggesting  that  as  the  percentage 
of  MPT  words  increased,  the  likelihood  of  a  GAO  total  program  over  budget 
decreased.  For  GAO  program  unit  cost  increase,  MPT  word  percentage  was 
a  significant  predictor  ( B  =  -1.05,  p  =  .040,  OR  =  0.35),  suggesting  that  as  the 
percentage  of  MPT  words  increased,  the  likelihood  decreased  for  a  GAO 
program  unit  cost  increase.  For  Weapon  Book  amount  spent  over  budget, 
ESOH  word  percentage  was  a  significant  predictor  ( B  =  5.23,  p  =  .067,  OR 
=  187.23),  suggesting  that  as  the  percentage  of  ESOH  words  increased,  the 
likelihood  of  a  Weapon  Book  amount  spent  over  budget  increased.  Also  for 
Weapon  Book  amount  spent  over  budget,  HFE/Hab/Surv  word  percent¬ 
age  was  a  significant  predictor  ( B  =  -4.90,  p  =  .006,  OR  =  0.01),  suggesting 
that  as  the  percentage  of  HFE/Hab/Surv  words  increased,  the  likelihood 
of  a  Weapon  Book  amount  spent  over  budget  decreased.  Last,  MPT  word 
percentage  was  a  significant  predictor  of  Weapon  Book  amount  spent  over 
budget  ( B  =  1.06,  p  =  .098,  OR  =  2.87),  suggesting  that  as  the  percentage  of 
MPT  words  increased,  the  likelihood  of  a  Weapon  Book  amount  spent  over 
budget  also  increased. 

Predictive  Equations 

For  each  of  the  four  significant  models,  each  regression  was  solved  to 
provide  a  predictive  formula  for  the  relationship  between  HSI  word  per¬ 
centage  outcomes  and  the  variables  of  interest  to  Hypothesis  One.  Each  of 
these  predictive  formulae  consider  ESOH  word  percentage,  HFE/Hab/Surv 
word  percentage,  or  MPT  word  percentage  as  subcategories  of  HSI  word 
percentage.  The  first  significantly  predictive  model  suggested  that  MPT 
word  percentage  was  the  only  factor  that  made  a  unique  contribution  to  the 
prediction  of  SAR  schedule  breaches.  Increased  percentage  of  MPT  words 
contributed  to  a  lower  likelihood  of  SAR  schedule  breaches.  This  model 
resulted  in  the  final  equation  shown  here. 

logit[Pr(Y=SAR  schedule  breach)]  =  0.06  -  0.74(MPT 

word  percentage) 

The  second  significantly  predictive  model  suggested  that  MPT  word  per¬ 
centage  was  again  the  only  factor  that  made  a  unique  contribution  to  the 
prediction  of  GAO  total  program  over  budget  cost  overruns.  Increased 
percentage  of  MPT  words  contributed  to  a  lower  likelihood  of  GAO  total 
program  over  budget  cost  overruns.  This  model  resulted  in  the  final  equa¬ 
tion  shown  here. 
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logit[Pr(Y=GAO  total  program  over  budget  cost  over¬ 
run)]  =  3.20  -  0.98(MPT  word  percentage) 

The  third  significantly  predictive  model  suggested  that  the  percentage  of 
MPT  words  was  again  the  only  factor  that  made  a  unique  contribution  to 
the  prediction  of  GAO  program  unit  cost  increases.  Increased  percentage 
of  MPT  words  contributed  to  a  lower  likelihood  of  GAO  program  unit  cost 
increases.  This  model  resulted  in  the  final  equation  shown  here. 

logit[Pr(Y=GAO  program  unit  cost  increase)]  =  2.61  - 
1.05(MPT  word  percentage) 

The  fourth  significantly  predictive  model  suggested  that  ESOH  word  per¬ 
centage,  HFE/Hab/Surv  word  percentage,  and  MPT  word  percentage  all 
made  a  unique  contribution  to  the  prediction  of  Weapon  Book  amount  spent 
over  budget  cost  overruns.  An  increased  percentage  of  ESOH  words  or  MPT 
words  contributed  to  a  greater  likelihood  of  Weapon  Book  amount  spent 
over  budget  cost  overruns,  while  an  increased  percentage  of  HFE/Hab/ 
Surv  words  contributed  to  a  lower  likelihood  of  cost  overruns.  This  model 
resulted  in  the  final  equation  shown  here. 

logit[Pr(Y=Weapon  Book  amount  spent  over  budget 
cost  overrun)]  =  -0.61  +  5.23(ESOH  word  percentage) 

-  4.90(HFE/Hab/Surv  word  percentage)  +  1.06(MPT 
word  percentage) 


Cost  overruns  identified  by  Weapon  Books  are  more 
affected  by  the  presence  ofHSI-related  terminology 
than  are  total  program  cost  overruns  identified 
by  GAO  assessments,  program  unit  cost  overruns 
identified  by  GAO  assessments,  and  schedule 
breaches  identified  by  SARs. 

Analysis  of  the  hypothesis  with  all  predictor  variables  yielded  the 
Nagelkerke  R2  values  for  the  four  significant  models:  SAR  schedule  breach, 
R2  =  0.06;  GAO  total  program  over  budget,  R2  =  0.13;  GAO  program  unit  cost 
increase,  R2  =  0.12;  and  Weapon  Book  amount  spent  over  budget,  R2=  0.14. 
Therefore,  cost  overruns  identified  by  Weapon  Books  are  more  affected  by 
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the  presence  of  HSI-related  terminology  than  are  total  program  cost  over¬ 
runs  identified  by  GAO  assessments,  program  unit  cost  overruns  identified 
by  GAO  assessments,  and  schedule  breaches  identified  by  SARs.  However, 
the  Nagelkerke  R2  value  is  low  for  each  of  these  four  outcomes,  thus  render¬ 
ing  little  predictive  power. 


Model  Sensitivity  and  Specificity 

Sensitivity  and  specificity  were  examined  for  each  model  using  classifi¬ 
cation  plots.  Each  plot  describes  the  percentage  of  correct  classifications  for 
a  predictive  equation.  Four  models  indicated  a  significant  predictive  abil¬ 
ity.  Therefore,  the  four  models  were  examined  for  their  ability  to  correctly 
classify  cases.  Results  of  the  classification  tables  are  presented  in  Table  8. 


TABLE  8.  CLASSIFICATION  TABLES  FOR  EACH  BINARY  LOGISTIC 

REGRESSION 


Dependent 

Observed 

Predicted 

Total 

Percentage 

Overall 

Variable 

No 

Yes 

Documents 

Correct 

Percentage 

SAR  Cost 

No 

352 

0 

100% 

Breach 

Yes 

156 

0 

508 

0% 

69% 

SAR 

No 

500 

0 

100% 

Performance 

Breach 

Yes 

9 

0 

509 

0% 

98% 

SAR 

No 

349 

6 

98% 

Schedule 

Breach* 

Yes 

141 

5 

495 

3% 

71% 

GAO  Total 

No 

16 

78 

17% 

Program 
over  Budget* 

Yes 

10 

188 

292 

95% 

70% 

GAO 

No 

4 

53 

7% 

Program 

Unit  Cost 
Increase* 

Yes 

2 

215 

274 

99% 

80% 

Weapon 

Book 

Amount 

No 

180 

4 

98% 

Yes 

53 

11 

248 

17% 

77% 

Spent  over 
Budget* 


Note.  An  asterisk  (*)  indicates  significance  at  the  p  <  .10  level. 
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As  shown  in  Table  9,  sensitivity  and  specificity  were  also  examined  using  clas¬ 
sification  plots  for  each  of  the  16  MDAPs.  Data  were  separated  by  MDAP,  and  16 
models  were  examined.  Nine  of  the  16  models  indicated  a  significant  predictive 
ability  and  were  examined  for  their  ability  to  correctly  classify  cases. 


TABLE  9.  CLASSIFICATION  TABLES  FOR  EACH  MAJOR  DEFENSE 

ACQUISITION  PROGRAM 

MDAP 

Observed 

Predicted 

Total 

Percentage 

Overall 

No 

Yes 

Documents 

Correct 

Percentage 

DDG  1000 

No 

1 

6 

14% 

Zumwalt 

Class 

Destroyer 

Yes 

2 

28 

37 

93% 

78% 

EA-18G 

No 

10 

6 

63% 

Growler 

Aircraft 

Yes 

0 

19 

35 

100% 

83% 

F/A-18E/F 

No 

8 

1 

89% 

Super 

Hornet 

Yes 

1 

2 

12 

67% 

83% 

F-22 

No 

6 

14 

30% 

Raptor 

Advanced 

Tactical 

Fighter 

Aircraft* 

Yes 

0 

51 

71 

100% 

80% 

Ground/ 

No 

3 

0 

100% 

Air  Task 

Oriented 

Radar* 

Yes 

0 

3 

6 

100% 

100% 

Joint  Land 

No 

10 

0 

100% 

Attack 

Cruise 

Missile 

Defense 

(JLACMD) 

Elevated 

Netted 

Sensor 

System* 

Yes 

0 

6 

16 

100% 

100% 

F-35  Joint 

No 

1 

5 

17% 

Strike 

Fighter 

Aircraft 

Yes 

0 

52 

58 

100% 

91% 
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TABLE  9. 

CLASSIFICATION  TABLES  FOR  EACH  MAJOR  DEFENSE 

ACQUISITION  PROGRAM  (CONTINUED) 

MDAP 

Observed 

Predicted 

Total 

Percentage 

Overall 

No 

Yes 

Documents 

Correct 

Percentage 

Joint 

No 

11 

5 

69% 

Tactical 

Radio 

System* 

Yes 

0 

29 

45 

100% 

89% 

Littoral 

No 

9 

4 

69% 

Combat 

Ship* 

Yes 

1 

13 

27 

93% 

82% 

Mine 

No 

11 

11 

50% 

Resistant 

Ambush 

Protected 

Vehicle 

Yes 

10 

16 

48 

62% 

56% 

P-8A 

No 

12 

2 

86% 

Poseidon 

Multi- 

Mission 

Maritime 

Aircraft 

Yes 

8 

0 

22 

0% 

55% 

Space 

No 

5 

4 

56% 

Based 

Infrared 

System 

High* 

Yes 

0 

39 

48 

100% 

92% 

V- 22 

No 

10 

6 

63% 

Osprey 

Joint 

Services 

Advanced 

Vertical* 

Yes 

5 

26 

47 

84% 

77% 

Vertical 

No 

3 

5 

38% 

Take¬ 
off  and 
Landing 
Tactical 
Unmanned 
Aerial 

Vehicle 

Yes 

4 

4 

16 

50% 

44% 

Warfighter 

No 

15 

11 

58% 

Information 

Network- 

Tactical* 

Yes 

6 

19 

51 

76% 

67% 
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Note.  An  asterisk  (*)  indicates  significance  at  the  p  <  .10  level.  An  additional  dependent 
variable  was  defined  from  the  six  dependent  variables.  If  there  was  any  breach  indicated 
by  any  of  the  dependent  variables,  then  a  value  of  1  was  assigned  to  this  new  variable.  If 
none  of  the  dependent  variables  indicated  any  breach,  then  a  value  of  0  was  assigned 
to  this  new  variable.  Because  the  Advanced  Deployable  System  (ADS)  had  no  breaches, 
as  indicated  by  the  six  dependent  variables,  the  new  variable  had  a  value  of  0  for  each 
case  (each  program  document).  Because  there  was  no  variance,  binary  logistic  regression 
analysis  could  not  be  applied  to  the  data  for  ADS.  There  were  seven  program  documents 
for  ADS,  and  those  seven  cases  have  been  omitted  from  this  table. 


Discussion 

In  reference  to  Hypothesis  One,  the  presence  of  HSI-related  words  in 
an  MDAP  document  may  be  associated  with  whether  or  not  a  program  will 
experience  a  schedule  breach  or  a  cost  overrun.  First,  assessing  individual 
predictors  from  three  significant  regression  models  suggests  that  a  schedule 
breach  identified  by  a  SAR  or  a  cost  overrun  identified  by  GAO  assessment 
of  total  program  cost  or  program  unit  cost  is  less  likely  to  occur  when  more 
MPT  words  are  present.  Second,  assessing  individual  predictors  from 
another  significant  regression  model  suggests  that  a  cost  overrun  identified 
from  Weapon  Book  budget  data  is  more  likely  to  occur  when  more  ESOH 
or  MPT  words  are  present.  However,  a  cost  overrun  is  less  likely  to  occur 
when  more  HFE/Hab/Surv  words  are  present.  Therefore,  considering 
SAR  data  and  GAO  assessment  data,  the  presence  of  terminology  about 
MPT  in  a  weapon  system’s  acquisition  program  documents  suggests  that 
the  program  might  not  experience  a  schedule  slippage  or  cost  overrun. 
Considering  Weapon  Book  budget  data,  the  presence  of  terminology  about 
MPT  in  a  weapon  system’s  acquisition  program  documents  suggests  that 
the  program  will  experience  a  cost  overrun.  Also  regarding  Weapon  Book 
budget  data,  the  presence  of  terminology  about  ESOH  in  a  weapon  system’s 
acquisition  program  documents  suggests  that  the  program  will  experience 
cost  overruns,  whereas  the  presence  of  terminology  about  HFE/Hab/Surv 
suggests  that  the  program  might  not  experience  a  cost  overrun. 

During  the  acquisition  life  cycle,  program  success  can  be  affected  by  various 
efforts  conducted  by  program  managers  and  systems  engineers.  As  shown  in 
Figure  2,  three  milestones  within  the  Analysis  of  Alternatives  (AoA)  occur 
prior  to  Milestone  A,  where  a  group  of  concepts  are  identified  and  compared 
among  one  another  (Department  of  Defense  [DoD],  2015).  Milestone  A  is 
when  the  Risk  Reduction  Decision  is  made,  whereby  a  specific  concept  is 
selected  for  further  development  and  resources  are  committed  to  the  mat¬ 
uration  of  relevant  technology  (p.  7).  Milestone  B  is  when  the  Development 
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Decision  is  made,  and  contracts  are  awarded  for  producing  and  testing  the 
concept  (p.  7).  Milestone  C  is  when  Low -Rate  Initial  Production  of  the  con¬ 
cept  begins  (p.  7). 


FIGURE  2.  ACQUISITION  LIFE  CYCLE 


Need  Identification 

(DoD:  Materiel  Development  Decision) 


Solution  Analysis 


Risk  Reduction  Decision 
.(DoD:  Milestone  A) 


Technology  Maturation  and 
Risk  Reduction 


Development  J 
Decisions  ' 


Requirements  Decision  Point 
(DoD:  CDD  Validation) 


^evelopment  RFP  Release 

Development  Contract  Award 
a  (DoD:  Milestone  B) 


Development 


LEGEND: 

A  =  Decision  Point 

CDD  =  Capability  Development  Document 

RFP  =  Request  For  Proposal 


Production 

Decisions 


Initial  Production  or  Fielding 
^^MHestone  C)  ^ 

Low-Rate  Initial  Production  or 
Limited  Deployment  and  Operational  Test 


k  Full-Rate  Production/Full  Deployment 


Figure  2  illustrates  the  sequence  of  decision  events  in  a  generic  program. 

It  is  not  intended  to  reflect  the  time  dedicated  to  associated  phase  activity. 


Production  Deployment, 
and  Sustainment 


Disposal 


Meanwhile,  a  program’s  cost  estimation  can  be  impacted  during  the  time 
period  leading  up  to  Milestone  B.  By  Milestone  B,  70  percent  of  a  system’s 
life-cycle  cost  will  have  been  determined  by  design  decisions  regarding  the 
program’s  features  and  efforts  (Deitz,  Eveleigh,  Holzer,  &  Sarkani,  2013; 
General  Accounting  Office,  1981;  Zimmerman,  Butler,  Gray,  &  Rosenberg, 
1984).  After  errors  have  been  made  at  the  Milestone  B  decision  point,  repair¬ 
ing  the  errors  or  compensating  for  them  costs  between  three  and  10  times 
more  than  the  cost  of  the  original,  erroneous  efforts  (Deitz  et  al.,  2013). 

Program  managers  and  systems  engineers  can  apply  the  observations  from 
this  investigative  study  to  their  understanding  of  human  considerations  and 
what  impact  human  considerations  have  on  the  development  of  a  given  pro¬ 
gram.  Systems  engineering  includes  HSI,  and  HSI  can  be  incorporated  into 
the  content  of  program  documents,  such  as  the  requirements  documents. 
Requirements  definition  is  one  facet  of  SE,  which  is  the  interdisciplinary 
approach  for  developing  systems  (INCOSE,  2012).  MDAPs  employ  SE  to 
conduct  weapon  systems  acquisition  for  the  DoD.  To  minimize  weapon  sys¬ 
tems  acquisition  costs,  the  DoD  created  the  Better  Buying  Power  mandate. 
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which  has  identified  some  focus  areas  with  appropriate  initiatives,  such  as 
(a)  eliminating  requirements  that  lead  to  nonvalue-added  processes,  (b) 
improving  how  requirements  are  defined,  and  (c)  inhibiting  requirements 
from  changing  over  time,  in  other  words,  requirements  creep  (DoD,  n.d.). 


Conclusions 

The  data  for  this  investigative  study  were  representative  of  different 
customers  within  the  U.S.  Government  (Air  Force,  Army,  DoD,  Marine 
Corps,  and  Navy)  and  of  different  types  of  weapon  systems  (aircraft,  com¬ 
munications  network,  ground  vehicle,  ship,  etc.),  which  helps  ensure  validity 
of  the  findings  among  customers  and  weapon  systems.  This  investigative 
study  looked  back  at  existing  MDAP  documents  in  a  retrospective  content 
analysis  as  a  means  to  look  forward  for  program  success.  Considering  how 
many  MDAPs  exist,  the  sample  size  was  relatively  small.  However,  the  value 
of  this  study  is  that  it  has  revealed  a  trend  that  HSI  practitioners  already 
suspected  and  that  can  be  examined  further  by  investigating  more  programs 
with  more  documents.  Exposing  trends  by  looking  at  historical  data,  such  as 
how  HSI  impacts  weapon  systems  acquisition,  is  informative  for  planning 
and  developing  future  systems. 
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The  Defense  Acquisition 
Professional  Reading  List  is  intended 
to  enrich  the  knowledge  and  under¬ 
standing  of  the  civilian,  military, 
contractor,  and  industrial  workforce 
who  participate  in  the  entire  defense 
acquisition  enterprise.  These  book 
recommendations  are  designed 
to  complement  the  education  and 
training  vital  to  developing  essential 
competencies  and  skills  of  the  acqui¬ 
sition  workforce.  Each  issue  of  the 
Defense  Acquisition  Research  Journal 
will  include  one  or  more  reviews  of 
suggested  books,  with  more  available 
on  our  Web  site  http://dau.dodlive. 
mil/defense-acquisition-profession- 
al-reading-program. 


We  encourage  our  readers  to  submit 
book  reviews  they  believe  should 
be  required  reading  for  the  defense 
acquisition  professional.  The  books 
themselves  should  be  in  print  or 
generally  available  to  a  wide  audi¬ 
ence;  address  subjects  and  themes 
that  have  broad  applicability  to 
defense  acquisition  profession¬ 
als;  and  provide  context  for  the 
reader,  not  prescriptive  practices. 
Book  reviews  should  be  450  words 
or  fewer,  describe  the  book  and  its 
major  ideas,  and  explain  its  rele¬ 
vancy  to  defense  acquisition.  Please 
send  your  reviews  to  the  managing 
editor,  Defense  Acquisition  Review 
Journal  at  DefenseARJ@dau.mil. 


Featured  Book 


To  Engineer  is  Human:  The  Role  of  Failure  in 
Successful  Design 

Author:  Henry  Petroski 
Publisher:  Vintage 
Copyright  Date:  1992 
Hard/Softcover/Digital: 

Softcover,  272  pages,  http://www.amazon. 
com/To-Engineer-Is-Human-Successful/ 
dp/0679734163 

ISBN: 

ISBN-10:  0679734163 
ISBN-13:  978-0679734161 

Reviewed  by: 

Henry  Petroski,  Professor  of  Civil 
Engineering  and  History,  Duke  University 


January  2016 


Review: 

This  book  has  its  origins  in  the  basic  question:  What  is  engineering?  It 
sets  forth  the  premise  that  under. standing/chZiire  is  essential  to  understand¬ 
ing  and  achieving  success  in  engineering.  Fundamentally,  engineering  is 
figuring  out  how  things  work,  solving  problems,  and  finding  practical  uses 
and  ways  of  doing  things  that  have  not  been  done  before.  Successful  engi¬ 
neers  properly  anticipate  how  things  can  fail,  and  design  accordingly.  Case 
studies  of  past  failures  thus  provide  invaluable  information  for  the  design 
of  future  successes. 

Conversely,  designs  based  on  the  extrapolation  of  successful  experience 
alone  can  lead  to  failure,  because  latent  design  features  that  were  not 
important  in  earlier  systems  can  become  overlooked  design  flaws  that 
dominate  the  behavior  of  more  complex  systems  that  evolve  over  time.  This 
paradox  is  explored  in  To  Engineer  is  Human  in  the  context  of  historical 
case  studies,  which  provide  hard  data  to  test  the  hypotheses  put  forward. 
Among  the  historical  data  points  are  the  repeated  and  recurrent  failures 
of  suspension  bridges,  which  from  the  1850s  through  the  1930s  evolved 
from  John  Roebling’s  enormous  successes— culminating  in  the  Brooklyn 
Bridge— to  structures  that  oscillated  in  the  wind  and,  in  the  case  of  the 
Tacoma  Narrows  Bridge,  twisted  itself  apart  and  collapsed  in  1940.  Lessons 
learned  from  these  cases  and  others  are  generalized  to  apply  across  a  broad 
spectrum  of  engineering  structures  and  complex  systems.  They  also  help 
explain  why  failures  continue  to  occur,  even  as  technology  advances. 
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Statement  Required  by  the  Act  of  August  12, 1970 
Section  3685,  Title  39,  U.S.C.  Showing  Ownership, 
Management,  and  Circulation 


The  Defense  Acquisition  Research  Journal  (ARJ)  is  published  at 
the  Defense  Acquisition  University  (DAU),  Fort  Belvoir,  VA.  The  University 
publishes  four  issues  annually.  The  Director  of  the  Defense  Acquisition 
University  (DAU)  Press  is  Randy  Weekes,  the  Managing  Editor  of  the 
Defense  ARJ  is  Norene  Taylor,  and  the  publisher  is  the  DAU  Press.  All  are 
collocated  at  the  following  address: 

Defense  Acquisition  University 
Attn  DAU  Press  (Defense  ARJ) 

9820  Belvoir  Road,  Suite  3 
Fort  Belvoir,  VA  22060-5565 

Average  Number  of  Copies  Each  Issue  During  the  Preceding  12  Months: 


A.  Total  number  of  copies  printed  (net  press  run) . 3,438 

B.  Paid  and/or  requested  circulation: 

1.  Sales  through  dealers  and  carriers,  street  vendors,  and  counter  sales . 0 

2.  Mail  subscriptions  paid  and/or  requested . 3,298 

C.  Total  paid  and/or  requested  circulation . 3,298 

D.  Free  distribution  by  mail,  carrier,  or  other  means;  samples, 

complimentary,  and  other  free  copies . 140 

E.  Total  distribution . 3,438 

F.  Copies  not  distributed: 

1.  Office  use,  leftover,  unaccounted,  spoiled  after  printing . 80 

2.  Returns  from  news  agents . 0 

G.  Total . 3,518 

Actual  Number  of  Copies  of  Single  Issue  Published  Nearest  to  Filing  Date: 

A.  Total  number  of  copies  printed  (net  press  run) . 3,400 

B.  Paid  and/or  requested  circulation: 

1.  Sales  through  dealers  and  carriers,  street  vendors,  and  counter  sales . 0 

2.  Mail  subscriptions  paid  and/or  requested . 3,260 

C.  Total  paid  and/or  requested  circulation . 3,260 

D.  Free  distribution  by  mail,  carrier,  or  other  means;  samples, 

complementary,  and  other  free  copies . 140 

E.  Total  distribution . 3,380 

F.  Copies  not  distributed: 

1.  Office  use,  leftover,  unaccounted,  spoiled  after  printing . 20 

2.  Returns  from  news  agents . 0 

G.  Total . 3,420 
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Defense  ARJ  and  AT&L  have  become 
online-only  publications  for  individual  subscribers 

If  you  would  like  to  start  or  continue  a  membership  with  Defense  ARJ  or  AT&L, 
you  must  register  a  valid  e-mail  address  in  our  LISTSERV 


iTSlitary-l 


Online  presence  for  easier  use  on  mobile  and 
desktop  devices:  http:y7dau.dodlive.mil 


Printable  PDF  files  can  be  found  at 

http://www.dau.mil/publications 


ALL  READERS:  Please  subscribe  or  resubscribe  so 
you  will  not  miss  out  on  receiving  future  publications. 

Send  an  e-mail  to  and/or 

datlonlineac  ,  giving  the  e-mail  address  you  want 
us  to  use  to  notify  you  when  a  new  issue  is  posted. 


•  Please  type  “Add  to  LISTSERV”  in  the  subject  line. 

•  Please  also  use  this  address  to  notify  us  if  you  change  your 
e-mail  address. 


•XA 


Defense  ARJ  Guidelines 

FOR  CONTRIBUTORS 

The  Defense  Acquisition  Research  Journal  (ARJ)  is  a  scholarly  peer- 
reviewed  journal  published  by  the  Defense  Acquisition  University  (DAU). 
All  submissions  receive  a  blind  review  to  ensure  impartial  evaluation. 


IN  GENERAL 

We  welcome  submissions  from  anyone  involved  in  the  defense  acqui¬ 
sition  process.  Defense  acquisition  is  defined  as  the  conceptualization, 
initiation,  design,  development,  testing,  contracting,  production,  deploy¬ 
ment,  logistics  support,  modification,  and  disposal  of  weapons  and  other 
systems,  supplies,  or  services  needed  for  a  nation’s  defense  and  security,  or 
intended  for  use  to  support  military  missions. 

Research  involves  the  creation  of  new  knowledge.  This  generally  requires 
using  material  from  primary  sources,  including  program  documents,  policy 
papers,  memoranda,  surveys,  interviews,  etc.  Articles  are  characterized 
by  a  systematic  inquiry  into  a  subject  to  discover/revise  facts  or  theories 
with  the  possibility  of  influencing  the  development  of  acquisition  policy 
and/or  process. 

We  encourage  prospective  writers  to  coauthor,  adding  depth  to  manuscripts. 
It  is  recommended  that  a  mentor  be  selected  who  has  been  previously 
published  or  has  expertise  in  the  manuscript’s  subject.  Authors  should  be 
familiar  with  the  style  and  format  of  previous  Defense  ARJs  and  adhere 
to  the  use  of  endnotes  versus  footnotes  (refrain  from  using  the  electronic 
embedding  of  footnotes),  formatting  of  reference  lists,  and  the  use  of  des¬ 
ignated  style  guides.  It  is  also  the  responsibility  of  the  corresponding  author 
to  furnish  a  government  agency/employer  clearance  with  each  submission. 
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MANUSCRIPTS 

Manuscripts  should  reflect  research  of  empirically  supported  experi¬ 
ence  in  one  or  more  of  the  areas  of  acquisition  discussed  above.  Empirical 
research  findings  are  based  on  acquired  knowledge  and  experience 
versus  results  founded  on  theory  and  belief.  Critical  characteristics  of 
empirical  research  articles: 

•  clearly  state  the  question, 

•  define  the  methodology, 

•  describe  the  research  instrument, 

•  describe  the  limitations  of  the  research, 

•  ensure  results  are  quantitative  and  qualitative, 

•  determine  if  the  study  can  be  replicated,  and 

•  discuss  suggestions  for  future  research  (if  applicable). 

Research  articles  maybe  published  either  in  print  and  online,  or  as  a  Web- 
only  version.  Articles  that  are  4,500  words  or  less  (excluding  abstracts, 
references,  and  endnotes)  will  be  considered  for  print  as  well  as  Web  pub¬ 
lication.  Articles  between  4,500  and  10,000  words  will  be  considered  for 
Web-only  publication,  with  an  abstract  (150  words  or  less)  included  in  the 
print  version  of  the  Defense  ARJ.  In  no  case  should  article  submissions 
exceed  10,000  words. 
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Book  Reviews 

Defense  ARJ  readers  are  encouraged  to  submit  reviews  of  books  they 
believe  should  be  required  reading  for  the  defense  acquisition  professional. 
The  reviews  should  be  400  words  or  fewer  describing  the  book  and  its  major 
ideas,  and  explaining  why  it  is  relevant  to  defense  acquisition.  In  general, 
book  reviews  should  reflect  specific  in-depth  knowledge  and  understanding 
that  is  uniquely  applicable  to  the  acquisition  and  life  cycle  of  large  complex 
defense  systems  and  services. 

Audience  and  Writing  Style 

The  readers  of  the  Defense  ARJ  are  primarily  practitioners  within  the 
defense  acquisition  community.  Authors  should  therefore  strive  to  demonstrate, 
clearly  and  concisely,  how  their  work  affects  this  community.  At  the  same  time, 
do  not  take  an  overly  scholarly  approach  in  either  content  or  language. 

Format 

Please  submit  your  manuscript  with  references  in  APA  format  (author- 
date -page  number  form  of  citation)  as  outlined  in  the  Publication  Manual 
of  the  American  Psychological  Association  (6th  Edition).  For  all  other  style 
questions,  please  refer  to  the  Chicago  Manual  of  Style  (16  th  Edition). 

Contributors  are  encouraged  to  seek  the  advice  of  a  reference  librarian  in 
completing  citation  of  government  documents  because  standard  formulas 
of  citations  may  provide  incomplete  information  in  reference  to  govern¬ 
ment  works.  Helpful  guidance  is  also  available  in  The  Complete  Guide  to 
Citing  Government  Documents  (Revised  Edition):  A  Manual  for  Writers 
and  Librarians  (Garner  &  Smith,  1993),  Bethesda,  MD:  Congressional 
Information  Service. 

Pages  should  be  double-spaced  and  organized  in  the  following  order:  title 
page  (titles,  12  words  or  less),  abstract  (150  words  or  less  to  conform  with 
formatting  and  layout  requirements  of  the  publication),  two-line  summary, 
list  of  keywords  (five  words  or  less),  body  of  the  paper,  reference  list  (only 
include  works  cited  in  the  paper),  author’s  note  or  acknowledgments  (if 
applicable),  and  figures  or  tables  (if  any). 

Figures  or  tables  should  not  be  inserted  or  embedded  into  the  text,  but  seg¬ 
regated  (one  to  a  page)  at  the  end  of  the  text.  When  material  is  submitted 
electronically,  each  figure  or  table  should  be  saved  to  a  separate,  exportable 
file  (i.e.,  a  readable  EPS  file).  For  additional  information  on  the  preparation 
of  figures  or  tables,  refer  to  the  Scientific  Illustration  Committee,  1988, 
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Illustrating  Science:  Standards  for  Publication,  Bethesda,  MD:  Council  of 
Biology  Editors,  Inc.  Restructure  briefing  charts  and  slides  to  look  similar 
to  those  in  previous  issues  of  the  Defense  ARJ. 

The  author  (or  corresponding  author  in  cases  of  multiple  authors)  should 
attach  a  signed  cover  letter  to  the  manuscript  that  provides  all  of  the  authors’ 
names,  mailing  and  e-mail  addresses,  as  well  as  telephone  and  fax  numbers. 
The  letter  should  verify  that  the  submission  is  an  original  product  of  the 
author(s);  that  all  the  named  authors  materially  contributed  to  the  research 
and  writing  of  the  paper;  that  the  submission  has  not  been  previously  pub¬ 
lished  in  another  journal  (monographs  and  conference  proceedings  serve  as 
exceptions  to  this  policy  and  are  eligible  for  consideration  for  publication  in 
the  Defense  ARJ);  and  that  it  is  not  under  consideration  by  another  journal 
for  publication.  Details  about  the  manuscript  should  also  be  included  in  the 
cover  letter:  for  example,  title,  word  length,  a  description  of  the  computer 
application  programs,  and  file  names  used  on  enclosed  DVD/CDs,  e-mail 
attachments,  or  other  electronic  media. 


COPYRIGHT 

The  Defense  ARJ  is  a  publication  of  the  United  States  Government  and 
as  such  is  not  copyrighted.  Because  the  Defense  ARJ  is  posted  as  a  complete 
document  on  the  DAU  homepage,  we  will  not  accept  copyrighted  manu¬ 
scripts  that  require  special  posting  requirements  or  restrictions.  If  we  do 
publish  your  copyrighted  article,  we  will  print  only  the  usual  caveats.  The 
work  of  federal  employees  undertaken  as  part  of  their  official  duties  is  not 
subject  to  copyright  except  in  rare  cases. 

Web-only  publications  will  be  held  to  the  same  high  standards  and  scru¬ 
tiny  as  articles  that  appear  in  the  printed  version  of  the  journal  and  will  be 
posted  to  the  DAU  Web  site  at  www.dau.mil. 

In  citing  the  work  of  others,  please  be  precise  when  following  the  author- 
date-page  number  format.  It  is  the  contributor’s  responsibility  to  obtain 
permission  from  a  copyright  holder  if  the  proposed  use  exceeds  the  fair  use 
provisions  of  the  law  (see  U.S.  Government  Printing  Office,  1994,  Circular 
92:  Copyright  Law  of  the  United  States  of  America,  p.  15,  Washington,  DC). 
Contributors  will  be  required  to  submit  a  copy  of  the  writer’s  permission  to 
the  managing  editor  before  publication. 

We  reserve  the  right  to  decline  any  article  that  fails  to  meet  the 
following  copyright  requirements: 
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•  The  author  cannot  obtain  permission  to  use  previously  copy¬ 
righted  material  (e.g.,  graphs  or  illustrations)  in  the  article. 

•  The  author  will  not  allow  DAU  to  post  the  article  in  our  Defense 
ARJ  issue  on  our  Internet  homepage. 

•  The  author  requires  that  usual  copyright  notices  be  posted 
with  the  article. 

•  To  publish  the  article  requires  copyright  payment  by  the 
DAU  Press. 


SUBMISSION 

All  manuscript  submissions  should  include  the  following: 

•  Cover  letter 

•  Author  checklist 

•  Biographical  sketch  for  each  author  (70  words  or  less) 

•  Headshot  for  each  author  should  be  saved  to  a  CD-R  disk  or 
e-mailed  at  300  dpi  (dots  per  inch)  or  as  a  high-print  quality 
JPEG  or  Tiff  file  saved  at  no  less  than  5x7  with  a  plain  back¬ 
ground  in  business  dress  for  men  (shirt,  tie,  and  jacket)  and 
business  appropriate  attire  for  women.  All  active  duty  military 
should  submit  headshots  in  Class  A  uniforms.  Please  note: 
low-resolution  images  from  Web,  Microsoft  PowerPoint,  or 
Word  will  not  be  accepted  due  to  low  image  quality. 

•  One  copy  of  the  typed  manuscript,  including: 

°  Title  (12  words  or  less) 

°  Abstract  of  article  (150  words  or  less) 

°  Two-line  summary 
°  Keywords  (5  words  or  less) 

°  Document  excluding  abstract  and  references  (4,500  words 
or  less  for  the  printed  edition  and  10,000  words  or  less  for 
the  online-only  content) 
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These  items  should  be  sent  electronically,  as  appropriately  labeled  files,  to 
the  Defense  ARJ  Managing  Editor  at:  DefenseARJ@dau.mil. 
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CALL  FOR  AUTHORS 


We  are  currently  soliciting  articles  and  subject  matter 
experts  for  the  2016  Defense  Acquisition  Research  Jour¬ 
nal  ( ARJ)  print  year.  Please  see  our  guidelines  for  con¬ 
tributors  for  submission  deadlines. 


Even  if  your  agency  does  not  require  you  to  publish,  consider  these  career-enhancing 
possibilities: 

•  Share  your  acquisition  research  results  with  the  Acquisition,  Technology,  and  Logistics 
(AT&L)  community. 

•  Change  the  way  Department  of  Defense  (DoD)  does  business. 

•  Help  others  avoid  pitfalls  with  lessons  learned  or  best  practices  from  your  project  or 
program. 

•  Teach  others  with  a  step-by-step  tutorial  on  a  process  or  approach. 

•  Share  new  information  that  your  program  has  uncovered  or  discovered  through  the 
implementation  of  new  initiatives. 

•  Condense  your  graduate  project  into  something  beneficial  to  acquisition  professionals. 


ENJOY  THESE  BENEFITS 

•  Earn  25  continuous  learning  points  for 
publishing  in  a  refereed  journal. 

•  Earn  a  promotion  or  an  award. 

•  Become  part  of  a  focus  group  sharing 
similar  interests. 

•  Become  a  nationally  recognized  expert 
in  your  field  or  speciality. 

•  Be  asked  to  speak  at  a  conference 
or  symposium. 


We  welcome  submissions  from  anyone  in¬ 
volved  with  or  interested  in  the  defense  ac¬ 
quisition  process— the  conceptualization, 
initiation,  design,  testing,  contracting,  pro¬ 
duction,  deployment,  logistics  support,  mod¬ 
ification,  and  disposal  of  weapons  and  other 
systems,  supplies,  or  services  (including  con¬ 
struction)  needed  by  the  DoD,  or  intended  for 
use  to  support  military  missions. 


If  you  are  interested,  contact  the  Defense  ARJ  managing  editor  (DefenseARJ@dau.mil)  and 
provide  contact  information  and  a  brief  description  of  your  article.  Please  visit  the  Defense  ARJ 
Guidelines  for  Contributors  at  http://www.dau.mil/pubscats/Pages/ARJ.aspx, 


Defense  ARJ 

PRINT  SCHEDULE 


The  Defense  ARJ  is  published  in  quarterly  theme  editions.  All  submis¬ 
sions  are  due  by  the  first  day  of  the  month.  See  print  schedule  below. 


Author  Deadline 

Issue 

July 

January 

November 

April 

January 

July 

April 

October 

In  most  cases,  the  author  will  be  notified  that  the  submission  has  been 
received  within  48  hours  of  its  arrival.  Following  an  initial  review,  submis¬ 
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